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ABSTRACT 


This  research  first  examines  the  use  of  the  Midshipmen  Information  System 
(MIDS)  by  faculty,  staff  and  midshipmen  as  a  performance  measurement  tool  at  the 
United  States  Naval  Academy.  Specifically,  this  project  examines  how  Company 
Officers  use  MODS  to  measure  the  performance  and  development  of  the  midshipmen  over 
time,  what  metrics  they  believe  are  important  to  midshipmen  development,  how  current 
MIPS  functionality  meets  the  needs  of  end  users  and  recommendations  for  improvement 
of  the  overall  system.  Research  includes  interviews  of  faculty,  staff  and  midshipmen  on 
their  use  of  MIDS,  an  online  survey  given  to  all  Company  Officers  and  a  detailed 
analysis  of  current  performance  measurement  models  in  use  today.  Once  this  data  was 
collected,  a  system  capability  analysis  of  MIDS  and  the  Webintelligence  ad-hoc  query 
software  was  completed.  These  results  are  included  in  an  appendix  that  can  be  used  by 
all  Company  Officers  as  a  training  guide  to  ensure  more  effective  use  of  their  time. 

The  findings  of  this  research  allow  the  United  States  Naval  Academy  in  general 
and  the  Company  Officer  specifically  to  more  fully  understand  the  importance  of 
performance  measurement  in  continually  improving  the  development  of  midshipmen. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  United  States  Naval  Academy  (USNA)  is  a  unique  institution  in  that  its 
primary  mission  is  to  develop  midshipmen  morally,  mentally  and  physically.  This 
mission  is  to  provide  graduates  with  the  assets  they  will  need  to  excel  as  officers  in  the 
United  States  Navy  and  Marine  Corps.  In  order  to  know  if  the  Academy  is 
accomplishing  this  mission,  a  performance  measurement  tool  must  exist  to  ensure  that 
each  midshipman  is  performing  at  the  required  level.  Many  of  the  faculty  and  staff  at  the 
Naval  Academy,  specifically  Company  Officers,  utilize  the  Midshipmen  Information 
System  (MIDS)  as  the  primary  source  in  obtaining  quantitative  performance 
measurement  data  used  in  tracking  the  development  of  midshipmen. 

W.  Edwards  Deming  introduced  performance  measmrement  as  a  new  management 
technique  in  the  1950’s.  Historically,  organizations  measured  their  performance  by 
financial  status  and  not  by  any  type  of  internal  measurements,  which  limited  their  ability 
to  analyze  actual  performance  of  the  organization.  The  introduction  of  performance 
measurement  gave  managers  and  leaders  the  flexibility  to  endure  new  customer  demands 
of  better,  faster  and  cheaper  outputs  (Frost,  2000).  The  success  of  performance 
measurement  in  the  private  sector  prompted  the  U.S.  government  to  create  the 
Government  Performance  and  Results  Act  of  1993  (Balanced  Scorecard,  2001). 

The  Government  Performance  Results  Act  (GPRA)  of  1993  was  developed  fi’om 
an  increased  concern  by  taxpayers  for  more  efficiency  and  effectiveness  within 
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governmental  organizations  (GPRA,  1993).  The  act  specifically  established  the 
requirement  for  all  governmental  organizations  to  set  performance  goals  and  define  the 
level  of  performance  needed  to  achieve  these  goals.  The  United  States  Naval  Academy 
must  adhere  to  the  guiding  principles  of  the  GPRA  because  it  is  an  agency  under  the 
Department  of  the  Navy  (Larges,  2000). 

When  Deming  pioneered  the  idea  of  performance  measurement  in  the  1950’s, 
leaders  in  the  management  field  began  extensively  studying  the  art  of  performance 
measurement,  and  many  new  models  have  been  introduced.  Two  of  the  models  that  have 
had  significant  impact  on  the  public  sector  are  the  Balanced  Scorecard  (2000)  and  the 
Measurement  Linkage  Model  (Chang  &  DeYoung,  1995).  -  A  third  approach  to 
performance  measurement  is  Frost’s  Strategy-Based  Performance  Management  Model 
(2000).  This  model  offers  a  more  simplistic  view  of  performance  measurement  and 
incorporates  a  more  thorough  explanation  of  the  relationship  between  management  and 
leadership. 

The  United  States  Naval  Academy  has  always  had  some  type  of  performance 
measurement  tool  available  for  Company  Officers  to  use  in  conducting  their  job. 
Recently,  the  Midshipman  Information  System  was  brought  online  as  the  official 
midshipmen  database  system.  The  system  was  designed  to  enable  faculty,  staff,  and 
midshipmen  to  have  access  to  all  types  of  performance  based  information  of  midshipmen. 
Being  able  to  utilize  and  measure  this  information  is  key  to  the  success  of  Company 
Officers  specifically  and  the  Academy  in  general.  After  all,  continuing  to  improve  on  the 
development  of  graduates  at  the  Academy  is  paramount  to  a  successful  future. 
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Recently,  Leadership  Education  and  Development  (LEAD)  graduate  students 
have  conducted  three  separate  studies.  The  studies  have  examined  the  performance 
measurement  at  the  Academy  and  developed  software  in  an  attempt  to  give  Company 
Officers  the  best  performance  measurement  tools  to  enable  them  to  succeed  and  fully 
imderstand  their  responsibilities  at  the  Academy.  These  studies  have  attempted  to  be  a 
catalyst  in  introducing  the  leaders  of  the  Academy  to  the  importance  of  performance 
measurement  and  the  understanding  that  if  an  organization  wants  to  improve  then  it  must 
measure  performance  and  overall  improvement  on  a  continuous  basis  (Chang  and 
DeYoung,  1995). 

Performance  measurement  is  a  necessary  tool  for  each  Company  Officer.  It  will 
enable  them  to  be  successful  in  developing  midshipmen  to  the  standards  set  by  the 
Academy.  For  this  to  occur.  Company  Officers  must  fully  understand  the  system,  which 
they  were  meant  to  use.  They  must  know  the  baseline  of  where  they  are  to  start  in  this 
development  process.  They  must  be  given  direction  as  to  how  the  development  is  to  take 
place,  and  they  should  be  given  a  clear  mission  and  strategy  for  success. 


B.  OBJECTIVES/PURPOSE 

The  purpose  of  this  research  is  to  examine  the  use  of  the  currently  deployed 
Midshipmen  Information  System  (MIDS)  by  Company  Officers  towards  performance 
measmrement.  The  goal  of  all  of  this  is  to  maximize  the  usefulness  and  efficiency  of 
MIDS  as  a  performance  measmement  tool.  This  will  improve  the  effectiveness  of 
Company  Officers  in  dealing  with  midshipmen  data,  thereby  allowing  them  more  time 
for  personal  interaction  with  midshipmen.  As  a  part  of  this  overall  goal,  this  thesis 
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should  be  the  next  step  in  a  continuous  process  of  evaluating  and  improving  performance 
measurement  at  USNA  as  a  whole  and  the  MIDS  system  in  particular. 

The  three  principal  parts  of  this  research  are  a  detennination  of  which  metrics  for 
performance  measurement  are  appropriate  for  Company  Officers  to  use,  an  assessment  of 
the  capability  of  MIDS  to  measure  these  metrics,  and  an  evaluation  of  the  extent 
Company  Officers  actually  use  MEDS  to  track  these  metrics.  Comparing  the  results  of 
these  three  parts  will  lead  to  recommendations  for  improving  the  training,  documentation, 
and  design  of  MIDS  as  well  as  determining  future  areas  of  study  in  performance 
measurement. 

The  first  objective  of  this  thesis  is  to  review  previous  research  in  the  area  of 
performance  measurement  in  the  public  sector  in  general  and  at  the  United  States  Naval 
Academy  in  particular.  Second,  this  research  will  use  stakeholder  interviews  and  a 
survey  of  Company  Officers  to  analyze  how  they  currently  measure  performance  of 
midshipmen  utilizing  MIDS  and  how  well  the  system  and  its  supporting  training  and 
documentation  support  their  needs.  From  the  results  of  these  two  portions,  a  list  of 
performance  metrics  will  be  developed.  Finally,  the  existence  and  availability  of  data  in 
MIDS  to  support  tracking  these  metrics  will  be  evaluated.  Using  these  results, 
improvements  to  MIDS  itself  and  its  supporting  training  and  documentation  systems  will 
be  proposed  in  order  to  make  it  more  useful  and  accessible  to  the  Company  Officer. 

This  is  a  continuing  effort  to  assess  the  Midshipmen  Information  System  and  how 
information  is  applied  to  midshipmen  development.  Once  complete,  this  study  will  assist 
the  Naval  Academy  in  improving  the  development  of  midshipmen  by  ensiuing  that  the 
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performance  measurement  tools  used  by  the  faculty  and  staff  have  a  direct  impact  on  the 
mission. 

C.  BENEFITS  OF  STUDY 

This  study  will  enhance  the  performance  measurement  tools  available  to  the 
USNA  Company  Officer  directly  and  provide  the  foundation  for  an  ongoing  process  of 
improvement  in  this  area.  The  increased  use  of  LEAD  program  resources  to  study 
performance  measurement  will  reduce  the  strain  on  the  Naval  Academy  administration. 
This  will  assist  them  in  continuing  to  provide  faculty  and  staff  with  cutting  edge  tools  in 
the  development  of  midshipmen  while  building  a  positive  relationship  between  the  Naval 
Academy  and  the  Naval  Postgraduate  School.  By  analyzing  the  current  knowledge  base 
associated  with  the  Midshipmen  Information  System,  Academy  administration  can  offer 
concentrated  training  to  LEAD  program  students  prior  to  their  shift  to  the  Company 
Officer  role.  The  result  of  this  training  will  be  a  more  efficient  access  to  data,  reducing 
the  time  required  for  this  function  and  increasing  the  time  available  for  other  leadership 
functions. 

D.  RESEARCH  QUESTIONS 

The  fundamental  question  that  this  research  aims  to  answer  is  “How  can 
Company  Officers  at  the  United  States  Naval  Academy  utilize  MIDS  to  its  fullest 
capability  as  a  performance  measurement  tool  for  the  development  of  midshipmen?” 
The  following  supplemental  research  questions  detail  how  specifically  various  portions  of 
this  research  support  answering  the  primary  research  question. 
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1.  What  prior  research  has  been  conducted  on  using  management  information 
systems  as  performance  measurement  tools  at  the  United  States  Naval 
Academy?  What  were  the  relevant  conclusions  of  this  research? 

2.  What  lessons  learned  and  best  practices  are  there  in  the  area  of  performance 
measurement  in  the  public  sector?  How  can  these  be  applied  to  performance 
measurement  at  the  United  States  Naval  Academy? 

3.  How  is  MIDS  currently  being  used  by  Company  Officers  in  measuring 
performance  of  midshipman? 

4.  How  well  does  the  current  MIDS  training  and  documentation  support 
Company  Officer  needs?  What  improvements  can  be  made  in  these  areas? 

5.  What  performance  metrics  do  Company  Officers  consider  important  to  the 
overall  development  of  midshipmen? 

6.  Is  the  data  for  these  performance  metrics  present  in  MIDS?  Can  it  be 
accessed  by  the  Company  Officers  in  an  efficient  and  useful  form? 

The  first  two  questions  will  be  addressed  in  the  literature  review  portion  of  this  thesis. 

Questions  three,  four,  and  five  will  be  the  focus  of  the  interview  and  survey  portions. 

The  final  question  will  be  addressed  in  the  MIDS  system  capability  analysis. 


E.  SCOPE  AND  METHODOLOGY 

Scope  of  this  research  is  limited  to  quantifiable  performance  measmes  used  by 
Company  Officers,  and  is  primarily  focused  on  the  use  of  MIDS  for  measuring 
midshipmen  on  individual  and  group  performance.  The  scope  will  include:  (1)  a  review 
of  previous  research  in  the  areas  of  performance  measurement  in  general  and 
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performance  measurement  tools  at  the  United  States  Naval  Academy  in  particular,  (2)  an 
evaluation  of  the  current  use  of  MIDS  by  Company  Officers  for  performance 
measurement,  (3)  an  assessment  of  the  capability  of  MIDS  to  measure  the  metrics  that 
Company  Officers  consider  important.  The  research  will  conclude  with 
recommendations  for  various  enhancements  such  as  training,  documentation,  and  query 
design.  Also,  recommendations  will  be  made  for  further  research  and  implications  for 
the  continual  improvement  of  performance  measurements  and  performance  measurement 
tools  at  the  United  States  Naval  Academy. 

The  methodology  used  in  this  research  will  consist  of  the  following  steps: 

1.  Conduct  a  thorough  literature  search  of  books,  journals.  Naval  Academy 
instructions  relevant  to  the  application  of  performance  measurement. 

2.  Conduct  group  interviews  consisting  of  Company  Officers,  a  Battalion 
Officer,  Commandant’s  Staff  and  their  midshipmen  counterparts.  Topics  will 
include  how  Company  Officers  use  MIDS,  ideas  for  improvement  of  MIDS 
and  the  knowledge  level  of  these  stakeholders  about  MIDS. 

3.  Conduct  a  survey  of  Company  Officers  to  determine  how  they  measure  the 
performance  of  their  midshipmen  and  how  they  use  MODS  as  a  tool  to  that 
end. 

4.  Analyze  data  collected  fi'om  interview  sessions,  survey,  and  MIDS  usage  data 
fi-om  the  system  itself 

5.  Develop  a  list  of  performance  measurement  indicators  that  Company  Officers 
can  use  in  measuring  their  midshipmen. 
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6.  Analyze  the  MDDS  system  for  the  presence  of  data  required  to  measure  these 
indicators  and  how  accessible  this  information  is  to  Company  Officers. 

7.  Develop  recommendations  for  improving  the  documentation,  training,  and 
design  of  the  MIDS  system. 

F.  ORGANIZATION  OF  STUDY 

This  study  is  divided  into  five  chapters,  and  includes  two  appendices.  This  first 
chapter,  the  introduction,  describes  some  of  the  goals  and  reasoning  underlying  the 
research,  the  benefits  that  are  hoped  to  be  gained  fi-om  it,  and  provides  an  overview  of  the 
entire  project. 

Chapter  n  includes  a  review  of  the  previous  research  in  the  area  of  performance 
measurement  at  USNA,  a  brief  overview  of  three  general  models  of  performance 
measurement,  and  a  brief  description  of  the  xxse  of  management  information  systems  for 
performance  measurement  by  Company  Officers,  including  a  description  of  the  current 
MIDS  system. 

Specifics  of  the  research  methodologies  used  in  this  study  are  described  in 
Chapter  in.  It  explains  the  protocols  used  in  conducting  stakeholder  interviews,  the 
methods  used  in  developing  and  administering  the  survey  to  Company  Officers,  and  the 
procedures  used  in  conducting  the  MIDS  system  capability  analysis. 

Chapter  IV  reports  the  results  of  the  research  described  in  the  prior  chapter.  In  the 
first  portion  of  this  chapter,  major  themes  and  important  findings  firom  the  interviews  are 
presented  and  results  of  the  survey  are  examined.  The  second  portion  of  the  chapter 
details  the  conduct  and  results  of  the  MIDS  system  capability  analysis. 
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Chapter  V  concludes  the  research  by  summarizing  the  important  conclusions 
derived  from  the  previous  chapters.  It  outlines  suggestions  for  improving  the  application 
of  MIDS  by  Company  Officers  for  performance  measurement,  particularly  in  the  areas  of 
module  design,  training,  and  documentation.  Finally,  recommendation  for  future 
research  are  presented. 

The  appendices  contains  supporting  information  important  to  understanding  the 
research.  Appendix  A  consists  of  the  Company  Officer  survey  and  its  results.  Appendix 
B  is  the  procedural  guide  for  Company  Officers  produced  during  the  MIDS  system 
capability  analysis. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

W.  Edwards  Deming  introduced  performance  measurement  as  a  new  management 
technique  in  the  1950’s.  Historically,  organizations  measured  their  performance  by 
financial  status  and  not  by  any  type  of  internal  measurements.  These  companies  were 
limited  in  their  ability  to  collect  data,  see  performance  baselines  and  targets  or  obtain 
feedback.  Financial  status  was  uncomplicated  and  easily  recognized.  Performance 
measurement  gave  managers  and  leaders  the  flexibility  to  endine  new  customer  demands 
of  better,  faster  and  cheaper  outputs  (Frost,  2000).  The  success  of  performance 
measurement  in  the  private  sector  prompted  the  U.S.  government  to  create  the 
Government  Performance  and  Results  Act  of  1993  (Balanced  Scorecard,  2001). 

The  Government  Performance  and  Results  Act  (GPRA)  of  1993  was  developed 
from  an  increased  concern  by  taxpayers  for  efficiency  and  effectiveness  inside 
governmental  organizations.  This  act  foimd  “waste  and  inefficiency  in  Federal  programs 
that  undermine  the  confidence  of  the  American  people  in  the  Government...”  (GPRA, 
1993).  The  act  specifically  established  the  requirement  for  all  governmental 
organizations  to  set  performance  goals  and  define  the  level  of  performance  needed  to 
achieve  the  goals.  The  act  also  required  performance  goals  to  be  objective,  quantifiable 
and  measurable.  To  set  these  goals,  each  organization  is  required  to  submit  a  strategic 
plan.  Stakeholder’s  desires  drive  the  development  of  the  strategic  plan.  Finally,  the  plan 
is  required  to  cover  how  these  goals  are  measured. 
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This  chapter  examines  current  trends  in  performance  measurement  geared  toward 
public  sector  organizations  and  why  they  have  become  important  to  their  success.  It  will 
discuss  the  use  of  these  practices  in  the  public  sector  and  specifically  how  the  current 
practices  are  in  use  by  Company  Officers  at  the  United  States  Naval  Academy.  The 
chapter  will  summarize  the  findings  of  three  previously  written  theses  in  the  area  of 
performance  measurement  as  applied  by  Company  Officers  at  the  United  States  Naval 
Academy.  Finally,  this  chapter  examines  the. Midshipmen  Information  System,,  which  is 
the  performance  measurement  tool  used  by  Company  Officers  in  the  development  of 
midshipmen. 

B.  PERFORMANCE  MEASUREMENT 

Numerous  performance  measurement  models  exist  that  can  be  used  to  improve 
the  performance  of  both  private  and  public  organizations.  It  is  however  more  difficult  to 
distinguish  performance  results  in  the  public  arena  since  these  organizations  gauge 
success  from  their  mission  rather  than  a  financial  statement.  This  section  covers  four 
main  areas.  The  first  subsection  is  a  review  of  current  strategies  on  performance 
measurement  in  the  public  arena.  The  second  discusses  historical  and  current  application 
of  information  technology  to  performance  measurement  at  the  United  States  Naval 
Academy.  The  third  subsection  will  review  previous  theses  written  in  this  area,  discuss 
the  models  used  in  the  research  and  briefly  look  at  the  metrics  this  research  found 
important  to  midshipmen  development.  The  final  subsection  discusses  the  Midshipmen 
Information  System  currently  in  use  by  Company  Officers  for  measuring  the  performance 
of  midshipmen. 
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1.  Current  Performance  Measurement  Models 

Since  Deming  pioneered  the  idea  of  performance  measurement  in  the  1950’s, 
leaders  in  the  management  field  have  introduced  numerous  performance  measurement 
models.  Private  organizations  have  realized  that  by  collecting  and  analyzing  the  correct 
data  they  can  affect  their  financial  and  overall  status  both  internally  and  externally.  The 
Government  Performance  and  Results  Act  of  1993  institutionalized  performance 
measurement  for  government  organizations  and  required  organizations  to  create  strategic 
plans  and  measure  performance  in  completing  the  mission  (Boone,  Hagen,  &  Utroska, 
1999).  Two  of  the  models  that  have  had  significant  impact  on  the  public  sector  are  the 
Balanced  Scorecard  (2001a)  and  the  Measiurement  Linkage  Model  (Chang  &  DeYoimg, 
1995).  A  third  approach  to  performance  measurement  is  Frost’s  Strategy-Based 
Performance  Management  Model  (Frost,  2000).  This  model  offers  a  more  simplistic 
view  of  performance  measurement  and  incorporates  a  more  thorough  explanation  of  the 
relationship  between  management  and  leadership.  This  subsection  will  address  each  of 
these  models  as  they  present  an  exciting  approach  to  performance  measurement  in  public 
organizations  and  namely  the  United  States  Naval  Academy. 

(L  Balanced  Scorecard 

Drs.  Robert  Kaplan  and  David  Norton  developed  the  Balanced  Scorecard 
in  the  early  1990’s.  This  new  concept  gave  organizations  a  clear  understanding  of  what 
they  should  be  measuring  in  their  organization  “in  order  to  ‘balance’  the  financial 
perspective”  (Balanced  Scorecard,  2001a,  p.  1).  Traditionally,  organizations  only  looked 
at  the  financial  dimension.  The  Balanced  Scorecard  approach  is  multidimensional.  In 
both  the  private  and  public  sector  the  idea  is  the  same — ^accomplishing  goals  means 
producing  an  output  faster  and  cheaper.  In  tiie  public  sector,  specifically,  taxpayers  are 
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the  customer  and  suppliers.  The  taxpayer’s  money  plays  a  large  part  in  the  strategy  and 
mission  that  exists  in  public  organizations.  To  obtain  these  goals,  the  organization  must 
measure  all  aspects  within  the  organization  and  not  limit  its  focus  to  the  final  output. 
This  model  directs  organizations  to  look  at  four  different  perspectives:  (1)  learning  and 
growth,  (2)  the  business  process,  (3)  customer,  and  (4)  financial.  The  management 
system  centers  these  perspectives  around  the  organization’s  strategy  and  vision.  Analysis 
of  the  strategic  plan  allows  organizations  to  create  metrics  to  measure  performance  within 
the  four  areas  of  the  organization.  The  Balanced  Scorecard  model  is  Figure  II-l. 


Figure  II-l :  Balanced  Scorecard  Model  (After:  Balanced  Scorecard, 

2001a) 

The  organization  identifies  metrics  from  the  strategic  plan.  A  method  is 
created  to  transform  information  into  numerical  data.  The  company  utilizes  an 
information  data  system  to  collect,  display  and  analyze  the  information.  According  to 
Kaplan  and  Norton,  this  information  has  the  ability  to  provide:  (1)  current  status  of  an 
organization  from  a  variety  of  perspectives,  (2)  feedback  for  continuous  improvements. 
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(3)  feedback  on  performance  measurement  methods  and  what  metrics  are  beneficial  to 
track,  and  (4)  inputs  for  forecasting  models  for  decision  support  systems  (Balanced 
Scorecard,  2001b). 

b.  Measurement  Linkage  Model 

Chang  and  DeYoung  (1995)  designed  the  Measurement  Linkage  Model  to 
assist  work  groups  of  an  organization  in  creating  and  implementing  a  performance 
measurement  system.  The  basic  concept  is  organizations  “. .  .must  continually  improve 
the  quality  and  productivity  of  our  products  and  services  to  stay  ahead  of  the 
competition”  (p.  5).  Chang  and  DeYoung  state  that  organizations  can  effectively 
improve  only  that  which  they  can  effectively  measure.  The  model  consists  of  eight  steps 
that  guide  the  work  group  through  an  improvement  plan  from  development  to 
implementation  and  outcome.  These  steps  allow  work  groups  to  link  their  performance 
measvuements  to  that  of  the  larger  organization  through  key  result  areas  (KRA’s)  and  key 
result  indicators  (KRI’s).  As  outcomes  are  achieved,  the  work  groups  analyze  their  own 
performance  measurements  and  the  organization  monitors  its  overall  performance. 
Feedback  allows  the  organization  to  reassess  what  it  is  measxuing  and  adjust  for 
continuous  improvement.  The  Measurement  Linkage  Model  is  shown  in  Figure  11-2. 

Chang  and  DeYoung  (1995)  identify  three  critical  reasons  why 
organizations  should  measme  improvement.  The  first  reason  is  to  provide  focus, 
direction,  and  a  common  understanding.  A  well-designed  and  executed  strategic  plan  and 
organizational  vision  will  focus  the  organization  on  what  to  measure.  The  second  is  to 
provide  knowledge  for  making  better  decisions.  Understanding  the  process  of  the 
organization  allows  improvements.  Having  access  to  good  data  allows  managers  to  make 
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good  decisions.  Having  too  much  data  or  an  inability  to  transform  data  into  information 
can  result  in  “information  overload”  (Boone,  et.  al.,  1999).  Harbour  stated, 
“Increasingly,  companies  are  moving  from  management  by  opinion  to  management  by 
fact.”  (1997,  p.  8)  Having  access  to  good  data  is  allowing  this  to  occur  and  having  a 
successful  performance  measurement  system  in  place  provides  this  data.  The  final  reason 
organizations  should  measure  performance  is  to  provide  feedback  on  organizational 
improvement  efforts.  Without  obtaining  data,  an  organization  will  be  unable  to  identify 
improvements  in  their  processes.  This  data  gives  a  snapshot  status  of  where  an 
organization  is  and  how  far  it  has  to  go. 

Following  these  three  reasons  on  why  to  measure,  Chang  and  DeYoung 
produced  ten  strategies  for  a  successful  performance  measurement  system  (1995,  pp.  8- 
10). 

1 .  A  system  should  provide  information-rich  data  that  is  “actionable.” 

2.  A  system  should  contain  a  masterful  blend  of  both  efficiency  and 
effectiveness  indicators. 

3.  A  system  should  include  measures  that  focus  on  accomplishment,  reward- 
oriented  categories. 

4.  A  system  should  not  measure  A  and  hope  for  B. 

5.  Measures  should  be  easy  to  understand. 

6.  Managers  or  employees  should  be  accountable  for  measurernent  accuracy 
and  results. 

7.  A  work  group  should  only  be  accoimtable  for  measures  over  which  they 
have  control. 
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8.  Meastirement  information  should  be  analyzed  and  acted  upon  in  a  timely 
maimer. 

9.  Measures  should  be  cost-effective  to  collect. 

A  measurement  system  should  focus  on  continuous  improvement,  rather  than  just 
comphance  and  control. 

Chang  and  DeYoung  (1995)  developed  the  Measurement  Linkage  Model 

to  ease  the  measurement  process  and  aid  organizations  in  the  improvement  process.  It 

starts  with  a  clear  understanding  of  the  organization  direction  followed  by  the 

development  of  metrics  on  what  performance  areas  to  measure.  The  next  step  identifies 

starting  points  and  performance  targets.  Once  these  targets  are  known,  the  work  groups 

set  objectives  to  reach  the  targets.  Finally,  the  work  groups  analyze  the  collected  data, 

provide  feedback  and  adjust  for  a  continuous  improvement  process. 

c.  Strategy-Based  Performance  Measurement 

Frost  (2000)  developed  the  Strategy-Based  Performance  Management 

model  for  two  reasons:  (1)  to  provide  managers  a  practical  approach  to  performance 

measurement  and  give  them  the  ability  to  decipher  data  into  usable  information  and  (2)  to 

explain  how  leadership  plays  an  important  part  in  deciding  what  metrics  to  collect  with 

regard  to  the  strategies  and  goals  of  the  organization.  Frost  used  the  analogy  that  old 

metrics,  primarily  financial  measures,  were  “...more  like  a  rearview  mirror  tVian  a 

windshield  or  a  steering  wheel”  (2000,  p.  7).  Now,  leaders  understand  that  an 

organization’s  strategic  plan  is  file  “steering  wheel”  and  that  metrics  developed  from  this 

plan  are  the  key  to  success. 
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Frost  (2000)  identified  four  vital  roles  that  metrics  play  in  the  activities  of 
a  leader.  These  roles  are  reporting  performance,  making  decisions,  implementing 
strategy,  and  improving  performance. 

In  reporting  performance,  metrics  allow  the  leader  to  fully  imderstand  the 
value  of  the  measurements  and  guarantee  employees  receive  full  credit  for  the  goals  and 
missions  they  have  accomplished.  To  accomplish  this  task,  leaders  must  remember  three 
key  items;  graphs,  which  are  easily  recognized  and  understood,  consistency  with 
measurement  definitions,  and  comparatives  that  allow  the  organization  to  gauge  itself 
against  another  organization. 

Opinion-based  decision  making  in  management  is  no  longer  a  viable 
option  in  today’s  fast  paced,  technology  driven  economy.  Having  the  right  information 
extracted  firom  the  metrics  being  tracked  allows  the  leader  to  practice  fact-based 
management  by  utilizing  hard  numbers  to  measure  current  performance  and  to  set  desired 
performance  levels  (Harbour,  1997). 

The  leader  within  an  organization  has  the  responsibility  of  implementing 
strategy  and  ensiuing  that  employees  fully  imderstand  the  mission  and  goals  of  the 
organization.  Developing  metrics  that  will  properly  align  and  direct  activities  of  the 
employees  can  assure  success  in  all  areas. 

“You  can’t  improve,  what  you  can’t  (or  don’t)  measure.”  (Harbour,  1997, 
p.  1)  The  only  way  to  guarantee  what  you  are  doing  as  a  leader  is  correct  is  to  start  fi’om 
a  baseline  and  measure  the  outcome.  Metrics  supply  information  to  the  leader  that 
enables  him  to  get  the  job  done.  The  Strategy-Based  Performance  Management  Model  is 
shown  in  Figure  11-3. 
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DRIVERS  METRICS  LEADERSHIP  TASK  ORGANIZATIONAL 

IMPACTS 


Figure  11-3:  Strategy-Based  Performance  Management  Model  (From:  Frost,  2000,  p.  81) 


Deciding  what  measures  to  track  can  be  the  most  important  decision 
organizations  make.  Frost  (2000,  pp.  28-33)  identified  three  areas  that  should  be 
examined  when  deciding  on  these  metrics  and  the  following  four  “drivers”  behind  the 
creation  of  these  metrics: 

1.  Strategy 

2.  Mission 

3.  Goals 

4.  Stakeholders 

The  four  drivers  can  be  divided  into  two  areas.  Strategy,  mission  and 
goals  make  up  the  first  and  stakeholders  make  up  the  second  area.  The  metrics  identified 
must  correspond  to  where  the  organization  wants  to  go  and  how  it  wants  to  get  there. 
The  ideas  for  metrics  come  primarily  fi-om  an  organization’s  strategy;  however,  “Because 
strategy  documentation  sometimes  does  not  capture  all  that  is  important,  you’ll  also  want 
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to  check  your  mission  and  goals  as  additional  sources”  (Frost,  2000,  p.  28).  For  example, 
one  initiative  in  the  United  States  Naval  Academy’s  strategic  plan  is  to  support  “the 
ethical  development  of  current  and  future  leaders  at  the  Naval  Academy,  the  Navy, 
Marine  Corps,  Department  of  Defense  and  beyond”  (USNA,  2000,  p.  1).  Add  to  this,  the 
Academy’s  overall  mission  is: 

To  develop  midshipmen  morally,  mentally  and  physically  and  to 
imbue  them  with  the  highest  ideals  of  duty,  honor  and  loyalty  in  order  to 
provide  graduates  who  are  dedicated  to  a  career  of  naval  service  and  have 
potential  for  future  development  in  mind  and  character  to  assume  die 
highest  responsibilities  of  command,  citizenship  and  government  (USNA, 

2001a,  p.  1). 

These  examples  elicit  a  number  of  areas  that  could  be  measured  to  track  how  the 
Academy  is  doing  in  developing  midshipmen. 

Stakeholders  are  the  second  area  of  concern  for  choosing  what  metrics  to 
track.  This  group  can  have  a  large  imp^t  on  the  organization,  especially  in  the  public 
arena.  When  basing  success  on  the  mission  and  not  on  financial  outcome,  public 
organizations  can  find  it  difficult  to  identify  stakeholders,  decide  what  is  important  to 
each,  and  select  the  metrics  fi’om  this  information.  For  example,  it  is  difficult  to  identify 
stakeholders  at  the  United  States  Naval  Academy.  This  list  can  include:  taxpayers, 
congress,  U.S.  Navy  and  Marine  Corps,  midshipmen,  and  midshipmen’s  parents.  By 
identifying  stakeholders,  organizations  have  the  ability  to  develop  strategy  and 
subsequently  identify  the  metrics  to  promote  success. 

“Major  failures  in  business  come  not  so  much  fi'om  uiunet  goals,  as  firom 
lack  of  response  to  unforeseen  changes.”  (O.L.  Duff  in  Frost,  2000,  p.  33).  When  dealing 
“snoozing  alligators”  the  final  area  of  concern,  it  is  very  important  for  leaders  to 
understand  that  the  strategic  plan  or  mission  may  not  hold  everyfiiing  that  is  important  to 
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the  organization.  At  the  United  States  Naval  Academy,  the  environment  is  continually 
changing.  New  leaders,  such  as  the  Superintendent  and  the  Commandant  of 
Midshipmen,  continue  to  update  the  goals  of  their  subordinates  and  this  can  be  one 
example  of  why  it  is  important  to  be  on  watch  for  the  “snoozing  alligator”. 

Once  metrics  are  identified  and  the  processes  in  place  to  track  them, 
leaders  and  managers  have  the  ability  to  perform  the  four  key  roles  outlined  earlier. 
Performing  these  roles,  the  leaders  can  affect  the  organization  in  the  following  five  areas: 

1.  Rationalized  resources  allow  the  organization  to  understand  the  scope  of 
its  capabilities. 

2.  Clear  accountabilities  ensure  goals  are  set,  employees  understand  job 
descriptions,  performance  measurements  are  reviewed  and  rewards  are 
presented. 

3.  Aligning  effort  behind  a  structurally  sound  strategic  plan  will  ensure 
people  are  working  on  things  that  matter  to  the  success  of  the 
organization. 

4.  Efficient  processes  ensure  the  people  get  the  job  done  the  cheapest,  fastest, 
and  best  way  possible. 

5.  Future  capabilities  are  destined  for  success  when  leaders  and  managers 
continue  to  evaluate  the  strategic  plan  and  performance  measurements 
(Frost,  2000,  pp.  35-52). 

Dr.  Frost  designed  the  strategy-base  performance  management  model  for 
executives  and  managers  attempting  to  improve  their  organization  with  the  latest  in 
performance  management.  He  places  great  emphasis  on  the  importance  of  metrics  and 
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leadership.  Not  only  should  these  metrics  present  good  data,  but  also  the  leaders  should 
be  able  to  use  the  information  extracted  from  the  data  to  improve  the  performance  of  the 
entire  organization. 

d.  Summary  of  Models 

The  three  models  addressed  show  the  importance  strategic  planning,  goal 
setting,  and  stakeholder  involvement  in  identifying  metrics  and  making  informed  fact- 
based  decisions.  The  Balanced  Scorecard  and  Measurement  Linkage  Model  were 
included  for  two  distinct  reasons.  The  first  is  that  both  models  have  been  successfully 
used  in  previous  studies  to  create  a  more  complete  understanding  of  performance 
measurement  at  the  Naval  Academy.  The  second  reason  is  both  models  comply  with  the 
directives  introduced  by  the  Government  Performance  and  Results  Act  and  they  provide 
clear  guidance  for  improving  the  overall  performance  of  public  organizations.  The 
Strategy  Based  Performance  Measurement  Model  was  included  because  it  incorporates 
leadership  as  an  integral  part  of  the  model  and  leadership  is  a  crucial  part  of  a  Company 
Officer’s  job  at  the  Naval  Academy. 

2.  Performance  Measurement  Application  at  USNA 

Performance  measurement  has  always  existed  at  the  United  States  Naval 

Academy.  In  order  to  rank  each  midshipman,  the  Academy  has  regularly  collected 
performance  data.  From  the  data.  Company  Officers  obtain  information  that  allows  them 
to  make  informed  decisions.  The  signing  of  the  Government  Performance  Results  Act 
(GPRA)  and  studies  completed  by  the  National  Partnership  for  Reinventing  Government 
made  performance  measurement  mandatory  and  required  the  public  sector  to  recognize 
and  abide  by  the  same  performance  measurement  and  management  standards  as  their 
civilian  counterparts  (Larges,  2000).  Two  performance  management  ^plications  have 
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been  created  that  allow  the  United  States  Naval  Academy  to  collect  the  data  to  be  used  in 
the  development  of  midshipmen.  These  applications  are  NATS  and  the  currently- 
employed  Midshipmen  Infomiation  System.  Two  portions  of  MIDS,  the  Company 
Officer  System  module  and  the  Company  Officer  -  Summary  Infonnation  module,  are 
designed  to  help  the  Company  Officer  with  performance  measurement  by  displaying 
various  midshipmen  information.  Discovering  how  Company  Officers  are  using  these 
tools  is  an  important  part  of  this  research.  COMIS  was  a  third  performance  management 
program  that  was  developed  for  use  by  the  Company  Officer  in  performance 
measurement,  however,  it  has  not  been  implemented  for  Academy  use.  The  following 
three  subsections  describe  these  applications  in  performance  measurement  management. 

a.  Naval  A  cademy  Time  Sharing  (NA  TS) 

The  NATS  system  was  the  first  computer-based  performance 
measurement  program  available  to  Company  Officers.  NATS  ran  off  a  Honeywell 
mainframe  that  was  particularly  limited  for  data  and  the  capability  to  extract  information 
(Cloughley,  W.  R.,  personal  communication,  January  2001).  These  limitations  would 
make  it  very  difficult  to  track  trends,  create  graphs  for  review  and  compare  performance 
of  midshipmen  or  entire  companies. 

b.  Company  Officer  Management  Information  System  ( COMIS) 

COMIS  was  designed  as  a  stand-alone  system  in  1999  prior  to  the 

Midshipmen  Information  System  going  online.  COMIS  was  designed  to  work  in 
conjunction  with  the  data  held  in  the  Midshipmen  Infoimation  System.  The  system 
allowed  Company  Officers  to  manipulate  the  data  into  practical  information  beneficial  to 
the  development  of  midshipmen.  COMIS  complied  with  all  aspects  of  the  GPRA  of 
1993  and  enhanced  the  Company  Officer’s  ability  to  improve  performance  (Larges, 
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2000).  In  order  to  folly  implement  the  system  a  database  manager  would  need  to  be 
identified  from  the  Information  Technology  Systems  Division  (ITSD)  or  Master’s 
program  (Larges,  2000). 

c.  Midshipmen  Information  System  (MIDS) 

Company  Officers  use  the  Midshipmen  Information  System  at  the  United 

States  Naval  Academy  as  their  only  online  performance  measurement  tool.  This  system 

and  the  metrics  used  to  track  midshipmen  development  is  the  crux  of  this  thesis.  The 

system  is  discussed  comprehensively  in  the  “Overview  of  MEDS”  section  of  this  chapter. 

The  metrics  used  by  Company  Officers  are  discussed  in  chapters  three  and  four. 

3.  Previous  Research  at  USNA 

The  commencement  of  the  Leadership  and  Education  Development  (LEAD) 
Masters  program  has  brought  an  abtmdance  of  attention  to  performance  measurement 
activities  at  the  United  States  Naval  Academy.  Company  Officers  have  completed  a 
plethora  of  research  in  the  general  area  and  more  specifically  with  the  tools  available  for 
them  to  complete  their  job.  This  research  has  always  been  to  advance  the  knowledge 
base  and  improve  the  overall  performance  of  both  midshipmen  and  Company  Officers. 
Three  theses  written  on  performance  measurement  at  the  academy  include:  “Performance 
Measurement  for  the  Company  Officer”  (Belz,  1999),  “A  Performance  Measurement- 
Based  Company  Officer  Management  Information  System  Prototype  for  the  United 
States  Naval  Academy”  (Boone,  Hagen,  and  Utroska,  1999),  and  “Analysis  of  the 
Company  Officer  Management  Information  System”  (Larges,  2000).  The  folloAving 
subsections  will  discuss  the  performance  measurement  models  used  in  this  research,  the 
metrics  identified  as  important  by  Company  Officers  in  the  development  of  midshipmen, 
and  summarize  the  findings  of  the  research. 
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a.  Belz  (1999)  “Performance  Measurement  for  the  Company 

Officer^' 

Belz’s  research  analyzed  the  extent  Company  Officers  use  performance 
measurement  in  the  development  of  midshipmen  (1999).  This  research  began  by  taking  a 
historical  look  at  the  roles  Company  Officers  have  had  in  leadership  and  management  of 
midshipmen.  This  research  involved  an  interview  and  questiormaire  of  Company 
Officers  to  look  at  the  different  techniques  used  in  measuring  performance  at  academy. 
From  the  interviews,  Belz  identified  sixteen  metrics  used  by  Company  Officers  to  assess 
their  midshipmen’s  and  overall  company’s  performance  (Belz,  1999): 

1.  Physical  Readiness  Test  (PRT)  scores 

2.  Physical  Education  (PE)  grades 

3.  Overall  Grade  Point  Average  (GPA) 

4.  Changes  in  GPA 

5.  Class  Absences 

6.  Extra  Instruction  (El)  hours 

7.  Number  of  D’s  and  F’s  in  military  performance 

8.  Number  of  semester  Academic  Board  cases 

9.  Number  of  Weight  Category  5  and  6  cases  (Belz,  1999,  p.  54) 

10.  Sick-in-room  chits  per  midshipmen  (Belz,  1999,  p.  55) 

11.  Attendance  at  company  functions 

12.  Morale 

13.  Success  in  intramurals 

14.  Drill  Grades 

15.  Drill  performance 

16.  Honor  offenses 
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Chang  and  DeYoung  (1995)  believe  that  each  Key  Indicator  (KI’s)  of 
performance  is  a  way  of  measuring  the  progress  of  an  organization.  Each  KI  should 
“...provide  critical/important  data,  be  easily  understood,  be  controllable  by  actions,  track 
actual  performance  change,  align  with  existing  data  or  be  clearly  estabhshed,  and 
measure  efficiency  or  effectiveness.”  (Change  and  De  Young,  2000,  p.  63).  By  looking 
at  the  mission’s  three  key  result  areas,  moral,  mental  and  physical  development,  Belz 
used  the  Measurement  Linkage  Model  to  identify  five  KI’s  fi-om  the  list  of  sixteen  shown 
above  that  clearly  met  each  requirement.  The  KI’s  so  identified  were  PRT  grades,  PE 
grades.  Overall  GPA,  Class  Absences,  and  Drill  Grades.  These  KI’s  were  tested  against 
three  key  areas  of  performance  measurement,  “data  collection  and  tracking,  baselines  and 
performance  targets,  and  effective  feedback”  (Belz,  1999,  p.  70).  This  breakdown 
concluded  that  only  overall  GPA  and  Absences  were  actually  used  in  the  performance 
measurement  sense  (Belz,  1999). 

b.  Boone,  Hagen,  and  Utroska  (1999)  Performance 

Measurement-Based  Company  Officer  Management  Information 
System  Prototype  for  the  United  States  Naval  Academy*’ 

Boone  et  al.  (1999)  developed  the  Company  Officer  Midshipmen 

Information  System  (COMES)  prototype  to  enhance  the  development  of  midshipmen  well 

into  the  next  century.  Chang  and  DeYoung’s  Measurement  Linkage  Model  was  utilized 

to  interview  Company  Officers.  These  interviews  had  parallel  findings  in  Key  Result 

Areas  (KRA’s)  and  key  indicators  (KI’s)  to  Belz’s  research.  Boone  et  al.  did  add  one 

additional  KRA,  professional  development.  This  research  loosely  identified  the  KI’s 

used  to  measure  the  performance  of  midshipmen  by  Company  Officers  in  each  KRA. 

The  KI’s  under  mental  development  included:  academic  grades,  number  of  honor 
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students  in  company,  academic  extra  curricular  activities  (ECA’s),  academic  boards, 
probation  and  UNSATS,  and  study  hours.  Physical  development  KI’s  included:  physical 
readiness  test  (PRT)  results,  PE  failures  and  marking  office  failures,  PE  curriculum 
grades,  weight  and  body  fat  standards.  Moral  development  KI’s  included:  number  of 
honor  and  counseling  offenses,  number  of  conduct  offenses  and  demerits,  amount  of 
community  involvement.  KI’s  under  professionalism  were:  absences/unauthorized 
absences,  professional  development  grades,  uniform  inspection/inspection  results,  room 
inspection  results,  4/C  professional  quiz/board  results,  and  performance  grades  (Boone  et 
al.,  1999,  p.51). 

After  identifying  the  metrics  needed  by  Company  Officers  to  succeed  in 
developing  midshipmen,  Boone  et  al.  conducted  a  system  requirement  and  analysis.  This 
analysis  looked  at  three  separate  areas:  performance  measures;  the  Midshipmen 
Information  System  Company  Officer  module;  and  Brigread  Plus  (Boone  et  al.,  1999). 
This  analysis  provided  the  information  lacking  from  the  Midshipmen  Information  System 
that  would  be  included  in  the  COMIS  prototype.  The  analysis  also  assisted  in  the  logical 
and  physical  design  of  the  prototype.  COMIS  was  designed  to  be  a  flexible,  user-friendly 
tool  that  utilized  the  latest  in  technology  and  that  could  interface  with  the  MODS  system. 

At  conclusion  of  the  research,  it  was  the  intent  of  the  authors  that  their 
findings  be  used  to  continually  upgrade  MEDS;  however,  that  has  not  been  the  case. 
Their  significant  findings  were  the  Professional  Development  KRA  and  the  KRI’s 
associated  with  it.  By  identifying  new  areas  to  measure  the  development  of  midshipmen, 
the  study  of  COMIS  could  be  a  catalyst  for  further  improvements  to  MIDS. 
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c.  Larges  (2000)  “Analysis  of  the  Company  Officer  Management 
Information  System^ 

Larges  (2000)  conducted  the  third  thesis  research  to  determine  the 
capability  and  feasibility  of  implementing  the  Company  Officer  Management 
Information  System  (COMIS).  The  COMIS  program  was  designed  utilizing  Chang  and 
DeYoxmg’s  Measurement  Linkage  Model  for  use  by  Company  Officers.  This  research 
included  a  survey  of  Company  Officers  that  investigated  the  requirement  for  a 
performance  measurement  tool  that  would  generate  the  information  required  by  the 
Company  Officers  from  data  stored  in  the  Midshipmen  Information  System.  The  survey 
concluded  that  COMIS  was  a  viable  performance  measurement  tool.  Company  Officers 
recommended  COMIS  be  incorporated  into  the  Midshipmen  Information  System  as  a 
separate  module,  fricoiporating  COMIS  into  the  Midshipmen  Information  System  would 
entail  the  need  for  a  database  manager  provided  by  the  Company  Officer  Masters 
Program  or  a  member  of  ITSD.  Finally,  a  feedback  mechanism  would  be  crucial  to  the 
success  of  this  database  manager  to  ensure  all  stakeholders  are  capable  and  efficient  in 
the  use  of  this  program  (Larges,  2000). 

This  study  exposed  the  critical  need  for  a  dynamic  performance 
measmement  tool  to  be  available  to  Company  Officers.  By  allowing  Company  Officers 
access  to  information  more  specific  to  their  needs,  they  can  complete  their  jobs  more 
efficiently  and  improve  the  overall  performance  and  development  of  their  midshipmen. 

tL  Summary  of  Prior  Theses 

Belz  (1999)  provided  a  basis  for  the  academic  study  of  performance 
measurement  at  USNA  by  reviewing  the  history  of  leadership  development  of 
midshipmen  and  introducing  performance  measurement  models  to  the  behavior  of 
Company  Officers.  Belz  used  the  Measurement  Linkage  Model  to  identify  sixteen 
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measures  of  midshipmen  performance.  Boone,  Hagen,  and  Utroska  (1999)  extend  this 
study  by  using  the  Measurement  Linkage  Model  as  the  basis  for  the  design  of  the  COMIS 
prototype.  Larges  (2000)  examined  the  COMIS  prototype  by  comparing  it  with  current 
performance  measurement  literature  and  conducting  a  survey  of  Company  Officers  to 
determine  how  useful  COMIS  would  be  to  them  as  a  performance  measurement  tool. 

4.  Overview  of  MIDS 

The  Midshipman  Information  System  (MIDS)  is  a  custom-designed  enterprise 
database  for  use  by  Naval  Academy  faculty,  staff,  and  midshipmen  (Cloughley,  W.  R., 
personal  communication,  January  2001).  MIDS  was  created  to  serve  as  a  repository  for  a 
wide  variety  of  midshipmen-related  data  for  the  use  of  the  faculty  and  staff  It  is  used  by 
the  academic  faculty,  members  of  the  Commandant’s  staff.  Company  Officers,  as  well  as 
other  assorted  staff  members  around  the  Academy. 

The  main  system  was  developed  entirely  in-house  by  members  of  ITSD,  while  the 
ad-hoc  query  functions  are  provided  through  a  third-party  data  mining  tool.  The  system 
was  initially  rolled  out  with  its  basic  functionality  in  January  1999.  Since  that  time, 
functionality  has  been  expanded  by  incrementally  adding  additional  components  as  the 
needs  of  the  users  and  the  resources  of  ISTD  has  allowed. 

Data  is  stored  in  an  Oracle  relational  database  that  runs  under  UNIX  on  three 
linked  Compaq  AlphaServers.  These  servers  are  accessible  firom  an5nvhere  inside  the 
Naval  Academy’s  intranet,  and  can  support  up  to  6,500  users.  Information  in  the  MIDS 
database  is  periodically  transferred  into  a  separate  data  warehouse  maintained  by  the 
Academy’s  Institutional  Research  Department,  adding  to  the  historical  record  of  academy 
information. 
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The  standard  user  interface  for  MK)S  is  a  collection  of  web-based  modules  that 


provide  access  to  pre-determined  subsets  of  the  data.  There  are  several  hundred  of  these 
individual  modules.  Each  module  is  typically  designed  for  a  specific  purpose,  such  as 
recording  class  absences,  viewing  midshipmen  grades,  or  maintaining  sports  and  club 
rosters.  Different  users  have  access  to  different  collections  of  modules  based  on  the 
individual’s  place  in  the  Academy  organization.  This  is  designed  so  that  users  have 
access  to  the  information  that  they  need  to  do  their  job,  while  preventing  unauthorized 
access  to  sensitive  personal  data. 

In  addition  to  the  standard,  web-based  interface,  there  is  another  access  method  to 
the  data  in  MIDS.  This  access  is  through  two  pieces  of  third-party  data  mining  software 
called  Webintelligence  and  InfoView.  Webintelligence  is  a  query-creation  tool  and 
InfoView  is  a  tool  that  is  used  to  view  these  queries.  This  software  allows  users  to  access 
all  data  contained  in  MIDS  through  a  variety  of  user-created  ad-hoc  queries.  These 
queries  are  piupose-built  reports,  typically  for  a  specific  user’s  unique  needs.  Queries 
can  be  constructed  using  the  Webintelligence  software’s  Java-based  interface  by  any  user 
who  has  access  to  the  system.  Since  this  access  allows  the  user  to  view  any  portion  of  the 
database,  and  because  of  the  high  cost  of  the  third-party  licenses,  only  a  few  select  users 
have  access  to  this  capability.  Among  those  groups  that  have  this  access  are  Company 
Officers,  Battalion  Officers,  and  Commandant’s  staff  members.  Webintelligence,  while 
very  powerful,  is  also  correspondingly  complicated  to  learn  and  use  effectively.  Because 
of  this,  although  it  is  possible  for  any  of  these  users  to  create  ad-hoc  queries,  in  practice 
ITSD  personnel,  responding  to  requests  from  these  users,  construct  most  of  these  queries. 
Queries  that  are  created  with  Webintelligence  can  also  be  viewed,  but  not  modified,  by 
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user  who  have  access  only  to  the  InfoView  portion  of  the  system,  which  includes  a  larger 
portion  of  the  Academy  staff.  Queries,  once  created,  can  be  shared  with  any  other  user 
that  has  InfoView  access,  allowing  one  person’s  work  to  benefit  others  arotmd  the 
Academy  (Business  Objects,  2000). 

Nearly  all  of  the  documentation  for  MODS  is  contained  in  online  help  web  pages 
accessible  from  any  of  various  MIDS  modules.  ITSD  recently  produced  a  pocket  guide 
to  the  MIDS  specifically  for  Company  Officers  (USNA,  2001b)  that  provides  basic 
information  on  how  to  gain  access  to  the  system  and  perform  some  basic  tasks.  Beyond 
this,  though,  all  other  information  on  system  use  is  contained  in  the  online  help  pages. 

C.  CHAPTER  SUMMARY 

This  chapter  reviewed  Chang  and  DeYoung’s  Measurement  Linkage  Model, 
Kaplan  and  Norton’s  Balanced  Scorecard  and  Frost’s  Strategy-Based  Performance 
Management  model  in  use  by  private  and  public  organizations.  The  foundation  of  each 
model  is  the  proper  development  of  measurement  indicators  firom  the  organization’s 
strategic  plan.  After  identifying  the  metrics,  leaders  and  managers  can  collect  data, 
obtain  first-rate  information  and  ultimately  improve  the  performance  of  the  organization. 

Information  technology  applications  used  at  the  United  States  Naval  Academy 
have  included  the  Naval  Academy  Time  Sharing  system  and  the  Midshipmen 
Information  System.  COMIS,  although  a  viable  prototype,  has  never  been  implemented 
as  a  stand-alone  performance  measmement  tool  or  incorporated  as  an  additional  module 
in  MIDS.  Nevertheless,  development  of  COMIS  has  highlighted  the  necessity  for  the 
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jBrequent  reassessment  of  Company  Officer  performance  measurement  needs  that  ensure 
the  successful  development  of  midshipmen. 

Belz  (1999)  used  Change  and  DeYoung’s  Measurement  Linkage  Model  to 
ascertain  what  performance  metrics  were  used  by  Company  Officers  at  the  United  States 
Naval  Academy.  Boone  et  al.  (1999)  also  used  this  model  to  identify  an  additional  KRA 
and  tie  metrics  to  all  KRA’s.  With  this  information.  Boon  et  al.  designed  the  COMIS 
prototype.  Continuous  study  in  the  area  of  performance  measurement  and  information 
systems  can  only  benefit  the  Academy,  Company  Officers,  and  the  overall  mission  of 
developing  midshipmen. 

The  remaining  chapters  of  this  research  will  focus  on  identifying  the  metrics  used 
by  Company  Officers  to  track  performance  of  midshipmen,  the  techniques  to  which 
Company  Officers  can  extract  that  information  from  MIDS,  and  attitudes  of  Company 
Officers  and  other  stakeholders  with  respect  to  MODS  and  performance  measurement. 
This  will  include  the  extent  to  which  Company  Officers  can  extract  trends,  graph  relevant 
outcomes  and  to  actively,  not  passively,  track  the  performance  of  their  midshipmen.  The 
next  chapter  describes  the  various  methods  used  in  this  research. 
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in.  RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

The  purpose  of  this  research  is  to  examine  the  currently  deployed  Midshipmen 
Information  System  (MIDS).  The  first  major  area  will  examine  current  trends  in 
performance  measmement  geared  toward  public  sector  organizations  and  why  they  have 
become  important  to  their  success.  This  area  will  also  assess  the  Midshipmen 
Information  System  currently  employed  at  the  United  States  Naval  Academy  to  identify 
the  metrics  being  used  by  Company  Officers  to  measure  the  performance  and 
development  of  midshipmen.  By  keeping  appraised  of  new  ideas  regarding  performance 
meastirement,  public  organizations  can  continue  to  meet  the  expectations  of  the 
stakeholders  and  fully  understand  their  goals. 

The  second  area  of  research  involves  gathering  data  from  the  end  users  of  the 
Midshipmen  Information  System.  Groups  of  Leadership  Education  and  Development 
(LEAD)  program  graduates,  currently  serving  as  Company  Officers,  will  form  the  basis 
for  evaluation  and  assessment  of  MIDS.  These  groups  will  also  be  involved  in 
generating  ideas  for  improvements  and  evaluating  prototype  enhancements  to  the  system, 
the  cooperation  and  participation  of  key  personnel  in  the  Academy’s  Information 
Technology  Systems  Division  (ITSD),  Company  Officers,  Battahon  Officers,  and 
Commandant's  Staff  is  crucial  to  the  success  of  this  and  follow-on  research  in  this  area. 
The  data  will  be  collected  primarily  as  part  of  a  class  project  for  the  Performance 
Measurement  class  in  the  LEAD  curriculum.  The  current  LEAD  program  students  will 
be  broken  into  four  groups.  The  four  groups  will  conduct  interviews  with  the  previously 
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mentioned  personnel.  Standardized  questions  will  be  asked,  followed  by  a  brainstonning 
and  open  disciission  session  on  the  benefits  of  the  system  and  possible  improvement 
areas.  Members  of  the  focus  group  will  be  solicited  for  ideas  to  improve  the  utility  of 
MIDS,  reasoning  behind  the  metrics  they  use  in  performance  measurements  and  the 
knowledge  level  of  Company  Officers  about  MIDS.  The  data  fi-om  these  interviews  will 
assist  researchers  in  focusing  the  questions  to  be  asked  in  the  online  sxuwey  to  be  given  to 
current  Company  Officers  as  part  of  the  next  step  of  the  research. 

The  third  area  of  research  involves  gathering  user  information  fi'om  Company 
Officers  who  are  the  primary  user  of  the  Midshipmen  Information  System  as  a 
performance  measurement  tool.  A  fifteen  question  survey  will  gather  feedback  in  the 
areas  of  metrics  currently  being  tracked  by  Company  Officers,  what  measurements  they 
feel  are  important  to  the  development  of  midshipmen  and  how  the  Company  Officers 
utilize  graphs  and  charts  for  easy  recognition  of  trends,  and  any  training  being  conducted 
on  MIDS  and  Web  Intelligence.  The  research  will  then  examine  if  the  implementation  of 
supplementary  training,  documentation,  and  design  enhancements  is  needed  to  ensure 
Company  Officers  have  the  knowledge  and  understanding  of  the  capabilities  of  the 
program. 

Analysis  of  interview  and  survey  responses  will  assist  in  accurately  summarizing 
the  current  use  of  MIDS,  identify  the  techniques  used  to  extract  usable  information  fi'om 
the  program  for  Company  Officers  to  be  successful  in  their  job  and  identify  any  future 
training  requirements  that  would  be  beneficial  to  the  Company  Officers  and  the 
institution.  This  process  will  assist  in  generating  conclusions  and  follow  on  research  to 
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guarantee  the  Midshipmen  Information  System  continues  to  be  updated  with  the  most 
current  ideas  in  performance  measurement. 

In  order  to  assess  the  validity  of  the  self-reported  information  in  the  survey,  the 
fourth  portion  of  the  research  performing  a  statistical  analysis  of  actual  usage  data 
generated  by  the  MIDS  software.  Two  specific  questions  will  be  answered  by  this 
statistical  analysis.  These  are  how  much  Company  Officers  use  MIDS  on  a  day-to-day 
basis  and  how  frequently  Company  OfBcers  access  various  portions  of  the  system. 

The  final  portion  of  the  research  will  focus  on  MIDS  itself.  Based  on  the  results 
of  interview  and  survey  data,  several  key  performance  metrics  will  be  selected  for 
analysis  of  MIDS  system  capability.  Various  portions  of ‘MIDS,  as  well  as  the 
Webintelligence  ad-hoc  query  tool,  will  be  examined  to  determine  the  most  effective 
methods  of  transforming  data  in  the  system  into  information  useful  for  tracking  selected 
metrics.  The  goal  of  this  portion  will  be  to  develop  a  simple  set  of  procedures  that 
Company  Officers  can  employ  to  analyze  each  of  these  metrics  regularly.  Also,  any 
Webintelligence  queries/reports,  spreadsheets,  etc.  required  by  these  procedures  will  be 
developed.  At  the  completion  of  this  research  these  procedures  and  supporting 
documents  will  be  supphed  to  ITSD  for  dissemination  to  Company  Officers.  Finally, 
possible  design  and  training  enhancements  discovered  in  this  phase  will  be  included  in 
the  recommendations  for  further  study. 

B.  REVIEWING  PERFORMANCE  MEASUREMENT 

The  initial  process  used  in  evaluating  the  Midshipmen  Information  System  entails 
thorough  review  of  ciurent  performance  measurement  models  being  utilized  by  public 
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sector  organizations.  A  number  of  performance  measurement  models  exist,  however  this 
research  will  review  only  three  models.  The  Balanced  Scorecard  is  presented  because  it 
has  been  increasingly  used  in  die  public  sector.  Chang  and  DeYoung’s  Measurement 
Lmkage  Model  is  included  since  it  has  been  used  extensively  in  previous  research 
conducted  at  the  United  States  Naval  Academy  to  establish  the  best  metrics  for  Company 
Officers  to  track  the  development  of  midshipmen.  Frost’s  Strategy-Based  Performance 
Management  model  is  included  because  it  incorporates  the  Balanced  Scorecard  model 
with  current  leadership  practices  and  provides  a  simple  and  easily  imderstood  approach  to 
performance  measurement. 

The  second  step  in  evaluating  MIDS  is  to  study  the  current  functionality  and  use 
of  the  program  by  Company  Officers.  This  information  will  identify  a  starting  point  for 
the  remainder  of  this  research.  By  determining  what  metrics  are  being  used  by  Company 
Officers,  the  research  can  isolate  these  metrics  and  evaluate  the  ease  of  which  they  can  be 
manipulated  into  information  useful  the  end  users.  This  data  will  be  collected  via  two 
somces.  First,  the  LEAD  program  students  will  conduct  group  interviews.  These 
interviews  will  include  battalion  staff,  faculty  and  midshipmen.  The  second  form  of  data 
collection  will  be  in  the  form  of  a  survey  given  to  Company  Officers. 

Lastly,  the  information  collected  on  the  metrics  will  enable  the  research  to 
identify  useful  training,  documentation  and  design  enhancements.  These  new  features 
will  make  program  use  easier  which  will  correlate  to  Company  Officers  having  more 
useful  information,  faster  access  and  subsequently  more  time  to  spend  in  a  mentor  role  to 
their  midshipmen. 
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c. 


INTERVIEW  COMPOSITION  AND  TOPICS 


The  puipose  for  this  interview  is  to  develop  a  thorough  understanding  of  the  way 
faculty,  staff  and  students  use  the  MIPS  system.  The  interviews  will  assist  in 
determining  what  performance  metrics  are  important  and  how  the  system  is  currently 
being  used.  Students  in  the  Leadership  Education  and  Development  program  will  lead 
the  group  interviews.  Each  group  will  conduct  two  interviews.  The  first  group  will 
interview  a  Battalion  Officer  and  two  midshipmen  assigned  to  the  Battalion  staff.  The 
second  group’s  first  interview  will  include  the  Performance  Officer,  Conduct  Officer  and 
a  civilian  employee  assigned  to  performance  measurement  on  the  Commandant’s  staff. 
Their  second  will  include  two  midshipmen  assigned  to  the  Commandant’s  staff.  The 
third  group’s  first  interview  will  include  four  currently  assigned  Company  Officers. 
Their  second  will  include  two  midshipmen  holding  the  position  of  Company  Executive 
Officer. 

During  these  interviews  the  LEAD  students  will  ask  a  number  of  predetermined 
questions  concerning  the  MIDS  system  and  how  it  is  used  by  each  of  the  interviewees. 
The  questions  include: 

1 .  What  metrics  are  important  in  measuring  the  development  of  midshipmen. 

2.  How  much  oftheassigmnentofmilitary  performance  grades  by  Company 
Officers  is  subjective  and  how  much  is  objective?  What  sort  of  process  do  you  go 
through  in  assigning  these  grades?  What  factors  influence  your  rankings?  How 
does  this  subjectivity  or  objectivity  affect  the  development  of  midshipmen? 
(Company  Officers  only) 
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3.  What  metrics  are  used  by  Company  Officers  (Battalion  Officers  or  Commandant 
staff)  in  measuring  performance?  Why  are  these  particular  factors  used?  Do  the 
midshipmen  know  what  these  are  and  understand  the  reasons  behind  them? 

4.  How  are  these  factors  recorded  and  how  is  that  data  used? 

5.  Does  MIDS  currently  provide  the  functionality  that  you  require? 

6.  What  new  functions  or  data  would  you  like  to  see  firom  MIDS? 

7.  Does  the  current  MIDS  documentation  (Pocket  Gmde,  online  help,  etc.)  meet 
your  needs? 

8.  Does  MIDS  training  support  your  needs?  If  not,  what  improvements  be  in  the 
areas? 

9.  What  measures  are  in  place  to  ensure  the  continued  relevance  of  MODS  to 
Company  Officer  performance  measurement  needs? 

D.  DEVELOPING  A  SURVEY 

As  mentioned  previously,  the  survey  is  used  to  collect  data  from  Company 
Officers  about  the  utility  of  Midshipmen  Information  System.  This  survey  is  designed  to 
collect  three  different  sets  of  data.  The  first  collects  data  on  metrics  Company  Officers 
feel  are  important  in  tracking  the  development  of  midshipmen.  The  second  collects 
feedback  on  Company  Officer  opinions  toward  functionality  and  iisage  data.  The  third 
set  attempts  to  identify  the  quality  of  training  and  documentation  that  currently  exist  with 
the  MIDS  system  and  what,  if  any,  improvements  can  be  made.  The  complete 
Midshipmen  Information  System  survey  is  included  in  Appendix  A. 
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1.  Survey  Format 

The  survey  will  be  taken  by  each  Company  Officer  via  the  Internet.  This  medium 
allows  the  Company  Officers  to  complete  the  survey  independently  without  interruption 
and  allows  them  access  to  the  MODS  system  if  needed  during  that  time.  The  survey 
includes  fourteen  questions  designed  to  identify  metrics  used  by  Company  Officers, 
training  and  functionality  of  the  program  and  any  improvements  that  should  be  made  to 
the  program. 

2.  Explaining  the  Survey  Questions 

Each  of  the  fourteen  question  is  designed  to  elicit  feedback  in  one  of  the  three 
areas  of  research  for  this  thesis.  Listed  below  is  each  question  and  the  reasoning  behind 
asking  each  of  them. 

1 .  Which  of  the  following  metrics  do  you  use  to  track  the  performance  of 
individual  midshipmen  or  your  company  as  a  whole?  (check  all  that 
^pply)  This  question  attempts  to  identify  trends  in  the  metrics  being  used 
by  Company  Officers  to  evaluate  their  midshipmen  for  performance 
grades. 

2.  Do  you  track  changes  in  performance  *over  time*  for  either 
individual  midshipmen  or  your  whole  company?  This  question 
determines  if  MIDS  is  being  used  to  track  performance  in  general  which 
would  identify  possible  problems  with  training  or  stand^ds  associated 
with  Company  Officer  responsibilities. 

3.  Do  you  use  any  visual  representations  (graphs,  etc.)  of  performance 
for  your  use,  your  company's,  or  your  Battalion  Officer's?  This 
question  determines  whether  Company  Officer  understand  the  relevance 
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of  visual  representation  of  performance  measurements,  how  they  can  be 
used  to  easily  show  trends  and  if  they  are  a  requirement  by  the  Company 
Officer’s  superiors. 

4.  Which  of  the  following  most  closely  describes  how  often  you  use  the 
MIDS  system?  This  question  is  answered  on  a  scale  firom  “once  a  week” 
up  to  “four  times  a  day”.  It  would  determine  the  likelihood  of  Company 
Officers  benefiting  firom  this  study  and  also  whether  they  understand  the 
importance  of  the  MIDS  system  as  a  performance  measurement  tool. 

5.  What  do  you  use  MIDS  for?  (Please  rank  in  order  of  importance.) 
This  question  allows  the  Company  Officers  to  rank  how  they  use  MEDS  in 
general  and  give  researchers  an  idea  of  where  the  Academy  could  benefit 
fi'om  increased  training  and  dociimentation.  The  answers  Company 
Officers  have  to  choose  fi-om  include: 

1 .  Track  the  performance  of  individual  midshipmen 

2.  Track  the  performance  of  your  company  as  a  whole 

3.  Produce  required  reports  for  your  Battalion  Officer 

4.  Gather  information  for  conduct/honor/  academic/performance  boards 

5.  Enter  midshipmen  performance  grade 

6.  Other  (please  state  below)  This  question  allows  Company  Officers  to 

list  any  other  uses  of  the  program  they  feel  is  important  to  accomplishing 
the  mission  of  the  Academy. 

7.  Which  module  of  MEDS  do  you  use  the  most?  This  question  is  used  to 
identify  what  modules  in  MIDS  are  used  the  most  by  Company  Officers. 
It  will  assist  the  researchers  in  identifying  the  training  associated  with 
each  module  and  developing  recommendations  for  additional  training  and 
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documentation  if  needed.  Qioices  to  be  provided  were  identified  fi-om 
usage  data  obtained  firom  the  Academy’s  Information  Technology  Systems 
Department  (ISTD),  charged  with  maintaining  the  MIDS  system. 
Company  Officers  can  select  fi'om: 

1 .  Company  Officer  Page 

2.  Company  Officer  -  Summary  Information 

3 .  Matrices  -  Query  Current  Midshipmen 

4.  Weekend  Eligibility 

5.  Absences 

6.  Other,  Please  Specify 

8.  Do  you  use  the  ad-hoc  query  system  (Webintelligence)? 

9.  If  so,  how  many  times  per  week?  These  questions  were  developed  fi’om 
the  interview  process.  Many  faculty  indicated  difficulties  in  using  this 
feature  and  the  questions  are  designed  to  determine  the  accuracy  of  these 
claims  and  to  identify  how  often  the  feature  is  used.  It  will  also  identify 
whether  trends  are  being  evaluated  since  this  is  a  major  fimction  of  the 
query  system.  The  Company  Officers  can  choose  fi’om: 

1.  None 

2.  Once 

3.  Three  times 

4.  Five  times 

5.  Seven  times 

6.  Nine  or  more 

10.  How  well  does  MIDS  functionality  meet  your  needs?  This  question 
reveals  to  what  degree  MIDS  supports,  and  is  a  useful  tool  for,  the 
Company  Officers.  The  Company  Officers  can  chose  fiom: 
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1.  Not  at  all 

2.  A  little 

3.  Somewhat 

4.  Mostly 

5.  Completely 

11.  What  new  functions  or  data  would  you  like  to  see  from  MIDS?  This 
question  is  used  to  identify  new  functions  not  currently  covered  by  a 
certain  module.  In  other  words,  the  data  is  in  the  system  but  is  difficult  or 
impossible  to  manipulate  into  useable  information. 

12.  How  well  does  the  current  MIDS  documentation  (Pocket  Guide, 
online  help,  etc,)  meet  your  needs?  This  question  reveals  to  what  degree 
how  useful  the  recently-published  pocket  guide  and  online  help  functions 
are  to  Company  Officers  in  using  the  MIDS  program.  Company  Officers 
can  chose  fi-om 

1.  Not  at  all 

2.  A  little 

3.  Somewhat 

4.  Mostly 

5.  Completely 

13.  How  well  does  current  MIDS  training  support  your  needs?  This 
question  is  similar  to  question  12,  but  it  is  addressing  whether  current 
training  is  adequate  for  Company  Officers  to  properly  use  the  system  to 
the  fullest  extent  possible.  Company  Officers  can  chose  fi*om 

1.  Not  at  all 

2.  A  little 

3.  Somewhat 
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4.  Mostly 

5.  Completely 

14.  What  improvements  can  be  made  in  these  areas  (training  and 
documentation)?  The  final  question  attempts  to  use  the  expertise  of  the 
Company  Officers  and  the  knowledge  gained  through  their  experience  and 
the  LEAD  program’s  Performance  Measurement  class  to  identify  any 
improvements  that  should  be  made  to  the  MIDS  program.  These 
recommendations  could  include  the  removal  or  addition  of  certain 
functions  within  the  program. 

E.  USAGE  DATA  ANALYSIS 

Information  gathered  via  survey  is  self-reported  and  thus  may  be  subject  to 
conscious  or  unconscious  bias  of  the  respondents.  Conscious  bias  in  the  responses  is 
unlikely  because  the  siarvey  given  for  this  project  is  voluntary,  the  information  collected 
is  not  personally  identifiable,  and  individual  results  will  not  be  divulged  to  the 
respondent’s  superiors.  In  order  to  provide  some  check  for  unconscious  biases,  two  of 
the  sxirvey  questions  will  be  cross-checked  with  usage  data  generated  by  the  MIDS 
software.  The  questions  selected  for  this  checking  are  those  that  lend  themselves  to 
straightforward  quantitative  verification.  Specifically,  responses  to  flie  questions  dealing 
with  the  frequency  of  overall  MIDS  usage  (question  #4)  and  the  most  commonly-used 
module  of  MIDS  (question  #6)  can  be  readily  compared  to  the  objective  usage  data. 

MIPS  generates  a  daily  log  of  system  usage  that  summarized  that  day’s  activities. 
This  log  is  broken  down  into  four  sections.  The  first  section  is  a  summary  of  the  number 
of  times  each  module  (component  part  of  MIDS)  was  accessed  by  midshipmen  and  by 
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non-midshipmen.  The  second  section  is  a  listing,  by  module,  of  which  users  accessed 
that  module  and  how  many  times.  The  third  section  repeats  this  data,  but  is  sorted  by 
user  instead  of  by  module.  The  final  section  hsts  the  total  number  of  times  each  module 
was  accessed  that  day. 

It  is  the  third  section  of  these  files  that  is  the  source  of  the  data  that  will  be  used  in 
this  analysis.  The  format  of  this  portion  of  the  data  files  is  the  simplest  form  fi-om  which 
to  separate  Company  Officer  usage  from  other  system  users.  Two  weeks  worth  of  usage 
data  (ten  working  days,  for  a  total  of  ten  data  files)  will  comprise  the  scope  of  this 
analysis.  First,  the  targeted  (third)  section  of  each  of  the  ten  files  will  be  singled  out  and 
combined  into  a  single  file.  This  data  will  then  be  filtered  to  select  users  that  are 
Company  Officers  and  remove  all  others.  The  processed  Company  Officer  data  will  then 
be  analyzed  using  various  statistical  and  spreadsheet  software  (SPSS  and  Microsoft 
Excel)  in  order  to  answer  the  desired  questions.  Finally,  the  calculated  results  of  this 
analysis  will  be  compared  to  the  reported  results  from  the  survey  and  a  qualitative 
assessment  of  the  validity  of  the  survey  data  can  then  be  made. 

F.  ANALYSIS  OF  MIDS  FUNCTIONALITY  FOR  TRACKING  SELECTED 

PERFORMANCE  METRICS 

The  final  portion  of  this  research  will  focus  on  the  ability  of  Company  Officers  to 
obtain  useful,  timely  performance  measurement  data  from  the  MIDS  database.  Based  on 
the  results  of  the  previously-conducted  interviews  and  survey,  the  metrics  considered 
most  important  by  Company  Officers  will  be  selected  for  analysis.  Both  MIDS  and  the 
Webintelligence  ad-hoc  query  tool  will  be  employed  to  determine  the  most  effective  way 
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for  Company  Officers  to  access  the  information  that  will  allow  them  to  track  these 
selected  metrics.  The  goal  of  this  section  of  the  project  is  to  provide  the  Company 
Officers  with  a  procedure  to  follow  in  assessing  each  of  these  metrics,  as  well  as  any 
supporting  Webintelligence  documents  or  other  files  needed  for  those  procedures.  These 
procedures  and  the  supporting  documents  could  be  distributed  to  the  Company  Officer 
as-is  or  could  be  the  basis  for  the  development  of  new  core  MIDS  capabilities. 

For  each  metric  in  question,  the  following  procedure  will  be  employed.  First,  the 
currently-available  modules  of  MIDS  will  be  examined  to  determine  what  information  is 
available  in  those  modules  how  it  relates  to  the  metric  in  question.  If  sufficient 
information  can  be  obtained  using  these  modules,  the  procedures  for  doing  so  will  be 
documented  and  the  analysis  will  move  to  the  next  metric. 

If  the  current  functionality  does  not  provide  an  adequate  way  to  measure  and  track 
the  metric  in  question,  the  Webintelligence  software  will  be  used  to  generate  an  ad-hoc 
query  document  that  retrieves,  analyzes,  and  displays  information  pertinent  to  the  metric 
in  question  in  the  most  effective  manner  possible.  If  analysis  beyond  the  capabilities  of 
Webintelligence  is  required,  desktop  software  such  as  SPSS,  Microsoft  Access,  or 
Microsoft  Excel  may  be  employed  to  work  with  data  extracted  firom  tiie  MIDS  database. 
Finally,  a  procedure  stepping  the  user  through  each  phase  of  the  analysis  will  be  created 
and  this,  along  with  any  queries,  spreadsheets,  or  other  files  will  be  compiled  in 
Appendix  B. 
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G.  CHAPTER  SUMMARY 


This  chapter  describes  the  five  areas  of  research  methodology  required  in  this 
thesis.  The  first  area  reviews  three  performance  measurement  theories  currently  in  use 
today.  The  second  area  consists  of  interviewing  faculty,  staff  and  midshipmen  to 
ascertain  their  current  use  of  the  system  and  what  they  believe  to  be  important  to  the 
development  of  midshipmen.  The  third  area  involves  gathering  information  in  the  forai 
of  a  survey  given  to  all  Company  Officers.  This  survey  will  identify  what  metrics  they 
feel  are  important,  how  they  use  the  system  to  do  their  job  and  their  opinions  as  to  the 
traimng  and  documentation  associated  with  the  program.  The  results  of  the  survey  are 
the  most  critical  element  of  the  research.  This  information  will  categorize  the  metrics 
being  used  by,  and  any  differences  between,  each  Company  Officer.  By  identifying  these 
differences,  the  research  will  be  able  to  recommend  future  action  to  Company  Officers  in 
utilizing  the  performance  measurement  data  to  ensure  all  midshipmen  are  evaluated  with 
the  same  set  of  metrics  in  the  fixture.  This  data  will  be  vitally  important  to  any  follow-on 
research  conducted  in  the  area  of  performance  measurement  at  the  Academy  as  it  sets  a 
baseline  of  current  practices.  The  fourth  area  of  research  complements  the  survey  by 
providing  an  objective,  quantitative  cross-check  of  its  validity  in  those  areas  where  an 
objective  analysis  is  possible.  This  is  accomplished  by  analyzing  actual  usage  data 
generated  by  the  MDDS  software  and  comparing  the  results  with  the  corresponding 
portions  of  the  survey.  The  final  portion  of  the  thesis  work  involves  assessing  the  ability 
of  MIDS  and  Webintelligence  to  provide  timely,  useful  information  to  the  Company 
Officer  for  use  in  tracking  selected  performance  metrics.  Currently-available  functions 
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will  be  assessed  and,  if  those  do  not  provide  all  of  the  required  information,  ad-hoc 
queries  will  be  created  to  fill  any  gaps  discovered  in  the  system. 

The  next  chapter  analyzes  and  summarizes  the  results  of  the  interviews,  survey 
and  usage  data  analysis,  as  well  as  generates  procedures  for  obtaining  data  from  the 
Midshipmen  Information  System  that  will  best  assist  Company  Officers  in  their  job.  The 
final  chapter  will  detail  recommendations  to  improve  the  performance  measurement  tools 
available  to  Company  Officers  so  that  they  are  up  to  date  with  the  most  current 
performance  measurement  practices  and  desires  of  the  Academy. 
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IV.  DATA  ANALYSIS 


A.  INTRODUCTION 

In  the  previous  chapters,  the  relevant  research  questions  for  this  thesis  were 
presented  and  developed,  the  previous  literature  relevant  to  the  subject  matter  was 
summarized,  and  the  methodology  used  in  this  research  was  described.  This  chapter 
continues  that  progression  by  covering  the  application  of  that  methodology  to  the  study’s 
research  questions  as  well  as  the  detailing  the  outcomes  of  that  work. 

The  chapter  is  divided  into  three  distinct  areas.  The  first  area  analyzes  the 
responses  given  during  group  interviews  conducted  by  LEAD  program  students  during 
the  Performance  Measurement  class.  During  the  interviews,  faculty,  staff  and 
midshipmen  answered  a  variety  of  informal  questions  regarding  their  use  of  the 
Midshipman  Information  System  (MIDS).  Answers  to  the  interview  questions  were  used 
in  generating  the  survey  that  was  disseminated  to  all  Company  Officers  via  the  Internet. 

The  second  area  analyzes  the  responses  given  to  the  survey.  Company  Officers 
answered  fourteen  questions  in  three  separate  areas  of  the  program:  (1)  which  metrics  are 
used  to  measure  performance,  how  they  are  used,  what  modules  are  used  and  to  what 
extent,  (2)  how  training  and  functionality  of  the  system  meet  current  needs  of  the 
Company  Officers,  and  (3)  recommendations  by  Company  Officers  for  improvements  of 
the  overall  system  to  increase  their  effectiveness  in  its  use. 

The  third  area  exploits  the  information  gathered  firom  the  interviews  and  survey 
by  developing  standardized  methods  for  extracting  the  data  firom  the  MIDS  database. 
The  methods  shown  will  assist  Company  Officers  in  using  their  time  efficiently. 
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Additionally,  it  will  aid  them  in  using  MIDS  as  a  more  effective  performance 
measurement  tool.  The  methods  are  shown  in-depth  in  Appendix  B. 

B.  GROUP  INTERVIEW  RESULTS 

As  mentioned  earlier,  LEAD  program  students  were  divided  into  three  groups  and 
each  group  was  to  conduct  two  separate  interviews  consisting  of  faculty,  staff,  or 
midshipmen.  The  first  group  had  two  interviews  scheduled:  the  first  included  one 
Battalion  Officer  and  the  second  (canceled  due  to  time  constraints)  would  have  included 
two  midshipmen  assigned  to  the  Battalion  Staff  The  second  group  interviewed  the 
Performance  Officer,  Conduct  Officer  and  a  civilian  employee  assigned  to  performance 
measurement  on  the  Commandant’s  Staff  in  their  first  interview.  Their  second  included 
two  midshipmen  assigned  to  the  Commandant’s  staff.  The  third  group  conducted 
interviews  with  foiu:  Officers  ciurently  assigned  as  Company  Officers  and  two 
midshipmen  holding  the  position  of  Company  Executive  Officer.  Each  interview  was 
designed  to  gain  insight  into  current  usage  of  MIDS,  training  and  documentation  of  the 
program  and  the  general  feelings  toward  the  program  by  Company  Officers  and 
midshipmen  alike. 

1.  Group  One  Interview  Results 
a.  Battalion  Officer 

The  Battalion  Officer  interviewed  has  been  m  the  position  for 
approximately  two  years.  She  stated  that  the  most  important  metrics  to  her  were  those 
that  had  a  reflection  on  the  morale  of  the  battalion.  These  included  absences,  conduct 
and  overall  grades.  She  stated  that  “we  use  MIDS  as  a  snapshot  of  the  condition  of  the 
battalion  but  not  in  any  other  way”  and  that  most  of  her  Company  Officers  did  not  have 
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the  requisite  knowledge  of  the  system  to  use  it  effectively.  This  statement  also  included 
her  own  abihty  to  manipulate  data  in  MIDS.  She  also  maintained  that  there  was  no  real 
way  to  measure  overall  moral  development  and  that  metrics  should  be  added  to  the 
program  functionality  to  measure  this  area.  The  Battalion  Officer  specifically  stated  that 
she  does  not  mandate  a  standard  for  metrics  that  should  be  tracked  in  MIDS  nor  in  the 
administering  of  performance  grades.  She  feels  that  performance  grades  are  given  out  in 
a  strictly  subjective  fashion  without  relying  on  data  in  MIDS  to  assign  grades. 

The  Battalion  Officer’s  responses  to  the  interview  questions  assisted 
greatly  in  the  generation  of  questions  relating  to  the  use  of  metrics,  what  Company 
Officers  felt  were  important  to  midshipmen  development,  and  to  training  issues. 

2.  Group  Two  Interview  Results 

a.  Commandant ’s  Staff Personnel 

This  interview  focused  primarily  on  performance  measurement  at  the 
Brigade  level  and  intended  to  assess  their  general  feelings  toward  the  use  of  MIDS.  The 
staff  members  identified  three  major  issues  associated  with  the  use  of  MIDS  as  a 
performance  measurement  tool  at  the  Academy.  The  first  was  measuring  performance  at 
the  macro  level,  the  second  was  the  MCDS  program  itself,  and  the  third  was  the 
ramifications  of  using  the  system  as  a  measurement  tool  at  the  Brigade  level. 

The  issue  of  tracking  performance  measurement  at  a  macro  level  was  that 
it  was  not  being  done.  One  interviewee  stated  that  “we  spend  a  great  deal  of  time 
investigating  areas  of  performance  within  the  organization,  but  they  do  not  measure, 
overall,  how  the  organization  is  accomplishing  their  mission”.  Additionally,  this 
interview  identified  the  feeling  that  individual/small  unit  analysis  was  being  conducted 
on  a  small  scale,  such  as  the  number  of  absences  assessed  to  a  particular  midshipman; 
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however  organizational  analysis  was  not  being  performed.  They  also  stated  that  no 
guidance  had  been  given  at  the  organizational  level  on  what  performance  metrics  were 
important  to  the  Academy  nor  on  how  various  metrics  were  linked  to  the  Academy’s 
mission  or  goals. 

The  second  issue  related  to  advantages  and  disadvantages  of  the  MIDS 
system  itself.  Advantages  to  the  system  included  user-friendliness,  a  good  source  of 
quantifiable  information,  and  quick  access  to  information.  Disadvantages  to  the  system 
included  a  weakness  with  comparisons,  a  lack  of  readily  available  training,  insufficient 
information  on  the  help  screen,  lack  of  depth  to  the  information  in  the  new  pocket  guide, 
a  distinct  lag  time  exists  between  data  entry  and  the  database  updating,  and  a  potential 
loss  of  personal  interface. 

The  third  issue  dealt  with  the  ramification  of  the  use  of  MIDS  at  the 
Brigade  level  for  evaluating  performance  measurement.  Persormel  interviewed  felt  that 
the  use  of  MODS  was  sort  of  a  “double-edged  sword”  in  that  there  was  a  definite 
communication  improvement  between  them  and  the  Company  Officer.  However,  they 
could  see  where  this  would  look  like  “big  brother”  is  always  watching  how  a  particular 
company  is  doing.  They  could  see  where  this  could  impact  the  decision  making  process 
of  the  Company  Officer. 

The  interviewees  identified  performance  and  conduct  as  two  areas  they 
felt  could  be  improved.  In  performance,  they  felt  the  inability  to  accomplish  comparison 
reports  on  each  company  was  detrimental  to  the  organization.  In  conduct,  they  identified 
the  abihty  to  track  loss  of  privileges,  restriction,  weekends,  and  special  request  chits  as 
important  to  the  Company  Officer  and  to  them  at  the  Brigade  level. 
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b.  Brigade  Staff  Midshipmen 

The  midshipmen  interviewed  held  positions  on  the  Brigade  staff.  They 
identified  four  areas  that  they  used  the  system.  These  included  the  generation  of  Form 
2’s  (conduct  reports),  attending  to  personal  records,  checking  their  Order  of  Merit  within 
their  class  (class  standing)  and  checking  class  absences  for  members  of  their  company. 
The  midshipmen  interviewed  had  the  opinion  that  Company  Officers  could  get  the  same 
information  “just  by  talking  to  the  midshipmen”  and  they  felt  many  midshipmen  saw 
MIPS  as  “leadership  by  computer”.  Overall,  the  midshipmen  interviewed  had  no  real 
understanding  of  MIDS  as  a  performance  measurement  tool  or  as  something  that  could  be 
used  to  improve  the  performance  of  midshipmen  and  the  United  States  Naval  Academy. 

Interviews  by  the  second  group  concluded  that  MIDS  as  a  whole  was  very 
useful  when  used  to  keep  track  of  data.  Currently,  they  felt  that  “it  is  not  used  to  great 
extent  by  the  Company  Officers  or  the  midshipmen”. 

3.  Group  Three  Interview  Results 
a.  Company  Officers 

The  Company  Officers  interviewed  consisted  of  two  LEAD  program 
graduates  and  one  post-Department  Head.  Overall,  the  Company  Officers  interviewed 
used  MODS  primarily  for  tracking  academic  performance,  conduct,  physical  fitness  and 
some  historical  data  extraction.  In  general,  all  agreed  that  little  to  no  direction  was  given 
to  them  regarding  how  MIDS  should  be  used  to  meet  the  goals  of  the  Academy.  They 
also  stated  that  every  company  uses  a  different  set  of  measurements  to  determine  the 
performance  grades  of  midshipmen  at  the  end  of  each  semester. 

With  regard  to  functionality,  the  Company  Officers  felt  that,  as  a  whole, 
MIDS  did  not  give  them  the  tools  they  required.  Specifically,  one  stated  that  the  “ad  hoc 
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query  function  could  be  very  useful,  however  there  is  no  training  and  the  queries  are  hard 
to  construct”.  Additionally,  they  had  been  exposed  to  training  and  documentation  for 
MIDS,  but  users  primarily  learned  only  what  they  needed. 

The  responses  to  the  interview  questions  supported  questions  regarding 
the  metrics  used  to  assess  midshipmen  performance,  training  and  documentation,  and  the 
overall  functionality  of  MEDS  given  to  Company  Officers  in  the  online  survey. 

b.  Company  Executive  Officer  Midshipmen 

The  two  midshipmen  interviewed  held  the  position  of  Executive  Officer 

within  their  company.  These  midshipmen  were  asked  the  same  general  questions  as  the 
officers  about  their  use  of  MIDS  in  relation  to  the  metrics  used,  training  and 
documentation  and  functionality  of  the  system.  Overall  the  midshipmen  interviewed  felt 
that  the  information  in  MIDS  had  no  bearing  on  performance  grades  and  the  only  utility 
the  program  had  was  looking  at  metrics  such  as  class  absences,  movement  orders  and 
academic  matrices.  Training  and  documentation  for  the  midshipmen  was  non-existent 
but  they  did  state  that  was  sufficient  for  their  limited  use. 

The  third  group’s  interviews  produced  two  important  recommendations. 
First,  separate  user’s  guides  should  be  created  for  the  Company  Officers  and  the 
midshipmen.  Additionally,  performance  standards  should  be  more  clearly  defined  across 
the  Brigade  along  with  a  standardized  performance  grade  assignment  policy.  This  would 
alleviate  some  of  the  subjectivity  in  the  assignment  of  performance  grades. 

C.  SURVEY  RESULTS 

Company  Officers  had  an  array  of  opinions  regarding  the  usefulness  of  MIDS  as  a 
performance  measurement  tool  in  evaluating  the  development  of  midshipmen.  The 
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fourteen  questions  presented  to  the  Company  Officers  attempted  to  evaluate  the  current 
use  of  MIDS,  what  metrics  Company  Officers  believe  are  important  in  midshipmen 
development,  and  assess  current  training,  documentation  and  fimctionality  of  the  system. 
This  section  presents  the  responses  given  by  the  Company  Officers  to  the  previously 
mentioned  survey  questions  along  with  any  comments  offered.  A  copy  of  the  complete 
survey  and  all  results  is  presented  in  Appendix  A. 

1.  Survey  Question  and  Response  Evaluation 

a.  Which  of  the  following  metrics  do  you  use  to  track  the 
performance  of  individual  midshipmen  or  your  company  as  a 
whole?  (Check  all  that  apply) 

As  mentioned  in  the  previous  section.  Company  Officers  for  the  most  part 
must  decide  individually  what  metrics  are  important.  Although  no  guidance  exists  as  to 
what  metrics  are  important  to  the  institution.  Company  Officers  are  definitely  capable  of 
making  these  decisions.  Unfortunately,  problems  may  arise  fi-om  differences  in  how 
performance  measurement  and  how  performance  grades  are  disseminated  in  each 
company.  Question  one  was  designed  to  allow  the  Company  Officer  the  ability  to  choose 
firom  a  fist  of  metrics  readily  accessible  on  MEDS.  The  question  also  allowed  the 
Company  Officer  to  add  any  metrics  they  tracked  and  deemed  important  to  their  job. 
Metrics  being  tracked  by  70%  or  more  of  the  Company  Officers  along  with  the  exact 
percentage  include:  PRT  scores  (88%),  overall  QPR  (96%),  changes  in  QPR  (72%), 
number  of  absences  (76%),  number  of  midshipmen  on  academic  probation  in  Company 
(80%),  munber  of  D’s  and  F’s  in  military  performance  (72%),  and  number  and  severity  of 
conduct  offenses  (88%).  Additional  responses  included:  conduct  history,  conduct  action 
pending,  and  movement  orders;  chain  of  command  input;  number  of  midshipmen  on 
merit  lists  (Deans,  Superintendent’s,  Commandant’s);  company  QPR  (semester  and 
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cumulative);  company  QPR  difference  (semester);  PE  deficiencies  (swimming,  PE 
failures,  etc.);  and  time  spent  talking  with  professors. 

It  is  important  to  note  that  each  of  the  twenty-two  metrics  listed  in 
question  one  received  at  least  one  vote  and  that  five  additional  responses,  citing  metrics 
not  on  the  list,  were  received  for  this  question.  These  responses  show  conclusively  that 
each  Company  Officer  has  a  somewhat  different  opinion  about  what  metrics  are 
important  to  the  successful  development  of  midshipmen.  Frost’s  states  that  “stakeholders 
are  the  second  major  source  for  choosing  performance  topics  and  that ,  Stakeholders  are 
simply  those  who  have  a  significant  stake  in  your  performance  and  an  abihty  to  create 
consequences  for  you,  good  or  bad”  (2000,  p.  30).  To  ensure  Company  Officers 
imderstand  the  mission  and  goals  to  midshipman  development,  guidance  and  direction 
fi'om  the  Academy  administration  and  other  stakeholders  in  choosing  metrics  must  occur 
and  is  crucial  to  success  in  meeting  the  needs  of  the  entire  organization. 

b.  Do  you  track  changes  in  performance  **over  time**  for  either 
individual  midshipmen  or  your  whole  company? 

Twenty-four  (96%)  Company  Officers  answered  positively  to  this 

question  while  one  (4%)  answered  negatively.  The  idea  of  tracking  changes  over  time  is 
successfully  explained  in  Frost’s  Strategy-Based  Performance  Management  Model.  The 
model  shows  that  by  tracking  metrics  leaders  are  capable  of  reporting  performance, 
making  mfoimed  decisions,  implementing  strategy  and  finally  improving  performance 
(Frost,  2000).  The  responses  given  by  Company  Officers  show  they  fiilly  understand  the 
importance  of  tracking  changes  in  performance  over  time  and  conduct  this  process  on  a 
continuous  basis. 
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c.  Do  you  use  any  visual  representation  (graphs,  etc.)  of 
performance  for  your  use,  your  company\  or  your  Battalion 
Officer’s? 

Ten  (40%)  Company  Officers  stated  that  they  did  utilize  visual 
representation  for  their  own  use,  their  company’s,  or  their  Battalion  Officer’s.  Sixteen 
(60%)  stated  that  they  did  not  utilize  any  form  of  visual  representation  of  midshipmen 
performance  for  their  own  use  or  for  others. 

Research  in  performance  measurement  suggests  that  this  attitude  is 
problematic.  For  example,  Harbour  stated  that  “having  hard  data  to  assist  an  organization 
in  their  planning  is  invaluable’’  (1997,  pp.  17-18).  Being  able  to  show  the  employees,  or 
in  the  case  of  the  Academy,  midshipmen,  where  the  institution  currently  stands  is 
likewise  invaluable.  By  doing  this,  all  personnel,  from  midshipmen  to  the  Commandant 
can  and  should  know  exactly  where  they  stand  and  where  improvements  can  be  made. 
Utilizing  trend  measurements,  charts,  graphs  and  comparative  charts  is  considered  by 
many  to  be  the  best  way  to  show  performance  information.  The  lack  of  visual 
representations  for  performance  metrics  is  a  significant  shortcoming  in  MIDS  that  will  be 
addressed  in  the  system  capability  analysis. 

d.  Hhich  of  the  following  most  closely  describes  how  often  you  use 
the  MIDS  system? 

This  question  was  answered  on  a  scale  firom  “once  a  week”  up  to  “four 
times  a  day”.  Two  (8%)  Company  Officers  stated  they  used  MIDS  once  a  day.  Nine 
(36%)  stated  they  used  MIDS  twice  a  day.  Fourteen  (56%)  stated  they  used  MIDS  four  or 
more  times  per  day.  The  responses  to  this  question  indicate  that  Company  Officers  are 
using  the  system  and  can  benefit  greatly  fi-om  a  study  in  performance  measurement  tools. 
It  also  identifies  an  understanding  by  Company  Officers  of  the  importance  of  MIDS  as  a 
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performance  measurement  tool.  This  question  identified  that  the  system  is  being  used 
and  the  next  question  explains  how  Company  Officers  are  using  the  system. 

Analysis  of  the  system  usage  data  generated  by  MIDS  supports  this 
conclusion.  For  the  two  week  period  studied,  the  Company  Officers  averaged  about  1 10 
page  hits  per  day  on  all  modules  of  MIDS.  As  can  be  seen  fi’om  the  P-P  plot  (Figure  IV- 
1),  the  variation  in  usage  was  approximately  normal.  The  variation  on  this  data  was  quite 
large,  with  a  sample  standard  deviation  of  92  hits/day  and  the  average  for  the  time  period 


Normal  P-P  Plot  of 
Average  #  of  Page  Hits/Day 


Observed  Cum  Prob 

Figure  rV-l :  Normality  Plot  for  MIDS  Module  Usage 

ranging  between  15  hits/day  for  the  least  firequent  user  to  360  hits/day  for  the  most 
fi’equent  user.  Although  ttiis  measure  of  page  hits  cannot  be  directly  translated  into  the 
number  of  sessions  that  a  Company  Officer  accesses  MIDS,  it  is  clear  that  Company 
Officers  are  using  the  system  quite  frequently  in  their  daily  routine. 
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e.  What  do  you  use  MIDS  for?  (Please  rank  in  order  of 
importance.) 

This  question  offered  Company  Officers  five  popular  uses  of  MIDS  to 
ascertain  how  fi-equently  they  conducted  each  of  them.  Uses  were  developed  firom  the 
group  interviews  conducted  before  survey  laimch.  The  follow-on  question  allowed  the 
Company  Officers  to  add  other  tasks  for  which  they  used  MIDS.  The  five  listed  uses  are 
shown  below  with  fi*equency  of  use  by  Company  Officers.  Nineteen  (76%)  Company 
Officers  responded  that  they  used  MIDS  to  track  the  performance  of  midshipmen  most 
frequently,  while  one  (4%)  responded  he  or  she  utilized  the  tracking  of  performance  the 
least  frequently.  Five  (20%)  Company  Officers  responded  they  used  MIDS  to  track  the 
performance  of  their  entire  company,  while  ten  (40%)  responded  they  used  the  system  for 
this  purpose  the  least  frequently.  Interestingly,  two  (8%)  Company  Officers  stated  the 
use  was  not  applicable  to  them.  Four  (16%)  responded  they  used  MIDS  to  create  reports 
for  their  Battalion  Officer,  where  seventeen  (68%)  responded  they  used  MEDS  least 
frequently  in  this  area.  Seventeen  (68%)  Company  Officers  used  MIDS  most  frequently 
to  gather  information  for  conduct/honor  and  academic/performance  boards.  Conversely, 
six  (24%)  stated  this  was  a  least  frequent  use.  Lastly,  one  (4%)  Company  Officer 
asserted  he  or  she  used  MIDS  to  enter  midshipmen  performance  grades  where  sixteen 
(64%)  answered  this  as  a  least  frequent  use. 

Seven  additional  uses  of  MIDS  were  listed  by  Company  Officers  which 

included: 

1.  Find  personal  information  (e.g.  major,  home  of  record,  sports  activities, 
contact  phone  numbers  for  leave  periods. 


61 


2.  Mostly  checking  individual  records  when  they  submit  chits,  checking 
absences  and  cross  referencing  them  with  Movement  Orders,  etc. 

3.  Approving  excusals/movement  orders,  finding  out  fi’ee  periods  so  I  can  talk  to 
one  of  my  Mids. 

4.  Initiate,  Review,  and  Approve  Movement  Orders  (MO)  and  Excusals. 

5.  MO/Excusal  approval;  Misc  INFO  re:  MDDN  [movement  order  and  excusal 
approval,  miscellaneous  information  about  midshipmen] 

6.  Gather  information  for  things  other  than  conduct/academic/etc,  boards. 

7.  Counseling,  spot-checking  and  MOs. 

The  uses  of  MIDS  listed  aided  the  research  in  focusing  efforts  to  show  the  most  efficient 
ways  to  manipulate  the  system  in  extracting  information  for  the  most  firequently  used 
fimctions. 

f.  Which  module  of  MIDS  do  you  use  the  most? 

The  Information  Technology  Systems  Division  (ITSD)  at  the  Naval 

Academy  has  developed  separate  modules  for  obtaining  performance  measurement  data 

on  midshipmen.  This  question  was  used  to  identify  which  modules  in  MIDS  were  used 

the  most  by  Company  Officers.  Choices  provided  were  identified  firom  usage  data 

obtained  jfrom  ITSD.  ITSD  personnel  recently  developed  the  Company  Officer  - 

Slunmary  Information  module.  This  module  was  designed  specifically  for  the  Company 

Officer’s  to  spend  their  time  more  effectively.  Responses  show;  however,  that  one  (4%) 

of  Company  Officers  actually  uses  this  module  as  their  most  firequent.  Twenty-two 

(88%)  utilize  the  Company  Officer  Page  module  as  their  most  firequent.  Two  (8%) 

“Other”  responses  were  given  which  included  the  ad-hoc  query  function  and  Movement 

Order/Excusal  pages. 
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Again,  actual  MIDS  usage  data  conforms  quite  closely  with  reported 
survey  results.  Table  IV- 1  shows  the  percentage  of  all  page  hits  for  the  two-week  time 
period  studied  that  were  in  each  of  the  listed  categories.  As  with  the  survey  response 
data,  the  Company  Officer  System  module  is  clearly  the  most  fi-equently  used,  gamering 
over  80%  of  all  page  hits  for  the  week.  This  closely  corresponds  to  file  88%  of  Company 
Officers  that  reported  they  used  that  module  most  fi-equently.  Interestingly,  in  the  case  of 
the  Company  Officer  -  Summary  Information  module,  usage  data  indicates  that 
Company  Officers  access  that  module  even  less  frequently  (less  than  0.1  percent  of  the 
time)  than  was  reported  in  the  survey. 

These  responses  specifically  show  that  almost  no  one  utilizes  the 
Company  Officer  -  Smnmary  Information  module,  which  a  large  amount  of  money  and 
time  was  spent  to  create. 


63 


Category 

% 

Company  Officer  System 

80.921% 

Movement  Orders 

6.188% 

Absences  &  Excuses 

5.696% 

Conduct  -  Record  Offenses 

3.519% 

Midshipmen  -  Query  General  Information 

1.570% 

Midshipman  -  Academic  Summary 

0.819% 

Summer  Striper  Billets  -  Assign  Midshipmen 

0.719% 

Midshipmen  -  Performance  Record 

0.181% 

Company  Officer  -  Summary  Information 

0.081% 

Summer  Training  -  Query  Assignments 

0.065% 

ECA's  and  Athletics 

0.053% 

Ac  Boards  &  COMAPs 

0.034% 

Section  List  -  Query  by  Specific  Course/Sect 

0.031% 

Provide  Security  for  Entering  Section  Grades 

0.016% 

Audit  -  Query  Relative  Information 

0.012% 

MIDS  Recent  Fixes  -  Query 

0.012% 

Midshipmen  -  Query  Academic  Information 

0.012% 

Schedules  -  Query  Midn  Schedule  for  Current  Semester 

0.012% 

Grades  -  Enter  by  Section 

0.009% 

Offered  Courses  and  Sections  -  Query 

0.009% 

Section  List  -  Query 

0.009% 

Summer  Striper  Billets  -  Maintain 

0.009% 

Display  Errors  From  File  Input 

0.006% 

Instructor  -  Query  Schedule 

0.006% 

PRT  Results  -  Query 

0.006% 

Table  IV-1:  Company  Officer  Usage  of  MIDS,  by  Module 


g.  Do  you  use  ad-hoc  query  system  (Webintelligence)  ? 

The  ad-hoc  query  system  is  capable  of  analyzing  data  and  presenting  the 

data  in  many  forms.  With  this  system,  Company  Officers  can  create  a  variety  of  visual 

aids,  such  as  graphs  and  charts.  As  mentioned  earlier,  showing  visual  aids  to  the 

midshipmen  helps  them  to  understand  what  performance  metrics  are  important.  It  sets  a 

baseline  for  where  they  are  currently  and  gives  the  midshipmen  direction  while  informing 

them  on  what  areas  they  need  work.  Twenty-three  (96%)  Company  Officers  stated  that 

they  use  the  ad-hoc  query  system  while  one  (4%)  stated  that  they  do  not  use  the  system. 

The  question  also  attempted  to  establish  the  number  of  times  per  week  the  query  system 

was  used  by  Company  Officers.  Responses  indicated  nine  (39%)  Company  Officers  used 
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the  system  once  per  week.  Twelve  (52%)  used  the  system  three  times  per  week.  One 
(4%)  used  the  system  five  times  per  week  and  one  (4%)  used  it  nine  or  more  times  per 
week.  Further  research  would  need  to  be  conducted  to  ascertain  the  current  specific  uses 
for  this  system.  With  the  previously  mentioned  data  on  the  number  of  Company  Officers 
using  graphs  and  charts  to  show  trends  in  their  company,  it  is  safe  to  say  that  this  is  not  a 
primary  use  of  the  ad-hoc  query  system. 

h.  How  well  does  MIDS  functionality  meet  your  needs? 

The  responses  to  the  question  address  the  ability  of  MIDS  to  perform  the 

required  fimctions  that  enable  Company  Officers  to  successfully  complete  their  job.  It 
does  not  however  address  what  functions  are  required  of  the  Company  Officers.  The 
responses  are  opinions  that  solely  indicate  to  what  extent  MIDS  functionality  meets 
Company  Officers’  personal  needs.  Responses  available  included:  not  at  all,  a  little, 
somewhat,  mostly,  and  completely.  Twenty-two  (84%)  Company  Officer  responses 
reveal  that  MIDS  functionality  “mostly”  meets  their  needs  while  three  (12%)  indicate  that 
MIDS  only  “somewhat”  meets  their  needs.  Surprisingly,  only  one  (4%)  indicated  that 
MIDS  functionality  “completely”  met  a  Company  Officer’s  needs  and  not  one  response 
indicated  “not  at  all”  or  “a  little”.  The  responses  show  the  importance  of  continuing  the 
improvement  process  of  MIDS.  In  fact,  certain  functions  may  already  exist  in  the  MIDS 
program;  however.  Company  Officers  have  not  been  given  training  or  direction  on  how 
to  best  utilize  the  system.  Therefore,  they  use  the  areas  of  the  system  that  they 
understand  and,  unfortunately,  these  may  not  be  identical.  This  shows  how  crucial 
Company  Officer  participation  is  in  creating  new  fimctions  in  MEDS  and  to  the  future  of 
MIDS.  The  follow  on  question  to  diis  identifies  new  functions  Company  Officers  would 
like  to  see  implemented  into  the  MODS  system. 
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L  What  new  functions  or  data  would  you  like  to  see  from  MIDS? 

Eighteen  responses  were  submitted  to  this  question  and  &e  complete  hst 

can  be  viewed  in  Appendix  A.  The  devotion  of  time  and  energy  required  for  the  ITSD  to 
make  changes  or  additions  to  the  MIDS  system  is  substantial.  Therefore,  substantial 
follow-on  research  on  these  recommendations  would  need  to  take  place  before  starting 
the  implementation  process.  The  Academy  administration  would  also  need  to  decide  if 
these  recommendations  are  congruent  with  the  strategies  and  goals  of  the  institution.  The 
following  are  a  few  of  the  recommendations  given  by  the  Company  Officers. 

1.  Develop  a  running  commentary  section  on  midshipmen  that  included  results 
and  problems  discussed  in  counseling,  performance  notes,  etc. . . 

2.  It's  great  that  we  pull  everything  fi'om  a  real-time  database,  but  I  would  like  to 
see  MIDS  work  from  a  cached  copy  on  my  computer.  The  vast  majority  of  the 
data  never  changes.  We  should  use  a  locally  cached  copy  as  the  default  and 
update  it  on  user  request.  This  would  work  well,  since  I  almost  always  look  at 
the  records  of  the  same  137  mids  (the  ones  in  my  company).  Also,  the  ad  hoc 
queries  should  be  simpler  and  more  intuitive  to  construct.  Very  few  people 
know  how  build  a  good  one.  The  laundry  list  of  corporate  documents  is  a  soup 
sandwich.  That  mess  should  be  significantly  streamlined. 

3.  I  would  like  to  see  the  MO  system  updated  to  be  more  user  friendly.  Mids 
often  come  to  see  me  because  they  don't  know  why  they  were  disapproved  for 
a  given  MO. 

4.  Ad  Hoc  queries  are  not  always  accurate  depending  on  when  and  where  the 
data  is  drawn.  This  often  leads  to  confusion.  I  would  like  to  see  more 
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functions  that  enable  me  to  look  at  my  company,  as  a  whole,  over  time. 
Company  Officer  -  Summary  Information  is  only  marginally  useful. 

5.  Graphs  for  trends  at  the  company  and  individual  level. 

The  recommendations  provided  show  that  most  Company  Officers  are 
significantly  concerned  with  performance  measurement  at  the  Academy.  They  seem  to 
understand  the  importance  of  continually  evaluating  the  metrics  being  tracked  and  also 
visually  showing  the  midshipmen  how  their  performance  is  comparing  their  own  past 
performance  and  that  of  their  peers. 

j.  How  well  does  the  current  MIDS  documentation  (Pocket  Guide, 
online  help,  etc.)  meet  your  needs? 

Documentation  on  MIDS  includes  the  recently-produced  pocket  guide  and 
the  on-line  help  functions.  The  pocket  guide  is  designed  to  introduce  system  capabilities 
and  access  procediures  to  end-users.  Some  of  the  procedures  included  in  the  guide  are 
how  to  access  MIDS,  how  to  get  help  on  the  system.  Commandant’s  staff  module 
capabilities.  Logistic  module  capabilities,  and  how  to  log  into  the  Webintelligence  ad-hoc 
query  tool.  Eight  (32%)  Company  Officers  indicated  that  this  docmnentation  did  not 
meet  their  needs  at  all  while  six  (24%)  indicated  that  the  dociimentation  met  their  needs  a 
little.  Another  six  felt  the  documentation  met  their  needs  somewhat.  Significantly,  only 
four  (16%)  Company  Officers  indicated  that  the  documentation  mostly  met  their  needs 
while  only  one  (4%)  indicated  that  it  completely  met  his  or  her  needs.  Overall,  fourteen 
of  twenty-five  (56%)  Company  Officers  indicated  that  MIDS  documentation  did  not  meet 
their  needs,  and  five  (20%)  indicated  that  it  met  their  needs. 

k.  How  well  does  current  MIDS  training  support  your  needs? 

The  research  conducted  has  discovered  that  no  organized  training  exist  for 

Company  Officers  to  learn  the  functional  capabilities  of  the  MIDS  system.  Training 
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received  on  the  system  includes  informal  training  given  by  ITSD  to  personnel  that  wish 
to  attend  their  training  sessions,  on  the  job  self-training  and  by  word  of  mouth  between 
end-users.  Fourteen  (58%)  Company  Officers  indicated  that  current  MTDS  training  did 
not  meet  their  needs  at  all  while  another  four  (17%)  indicated  that  the  training  met  then- 
needs  a  little.  Three  (13%)  Company  Officers  felt  the  training  met  their  needs  somewhat. 
Only  two  (8%)  Company  Officers  indicated  that  the  training  mostly  met  their  needs  while 
only  one  (4%)  indicated  that  it  completely  met  their  needs.  Overall,  eighteen  of  twenty- 
five  (75%)  Company  Officers  indicated  that  MIDS  training  did  not  currently  meet  their 
needs  and  three  (13%)  indicated  that  it  met  their  needs. 

L  What  improvements  can  be  made  in  these  areas  (training  and 
documentation)? 

The  survey  responses  show  conclusively  that  the  Company  Officers  use 
the  MIDS  system  for  a  variety  of  applications  centered  around  performance  measurement 
of  midshipmen.  This  question  attempted  to  determine  new  recommendations  Company 
Officers  would  like  to  have  implemented  in  the  area  of  training  and  documentation. 
Seventeen  recommendations  were  provided  and  are  listed  in  Appendix  A.  The  following 
summarizes  that  list.  The  most  common  recommendation  was  for  the  Academy  to 
provide  formal  training  on  the  functionality  of  MIDS.  This  would  enable  Company 
Officers  to  understand  exactly  what  data  is  available  and  how  the  data  can  be 
manipulated.  A  large  portion  of  the  responses  to  this  question  emphasized  the  point  that 
no  real  training  exists  and  that  most  Company  Officers  learn  the  system  by  “messing 
around  with  certain  modules  until  I  found  out  which  ones  were  most  useful  for  me”  and 
“on  the  job  training”.  Additionally,  training  in  the  area  of  the  ad-hoc  query  function  was 
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deemed  very  important  because  of  the  difficulty  experienced  in  creating  new  queries  and 
generating  graphs  and  charts  to  show  individual  and  company  wide  performance. 


D.  USING  MIDS  TO  MEASURE  PERFORMANCE 

This  portion  of  the  research  focuses  on  developing  easy-to-use  methods  of 
extracting  information  contained  in  the  MIDS  database  and  displaying  it  so  that  Company 
Officers  can  use  it  to  help  measure  the  performance  of  their  company  as  a  whole  and  their 
individual  midshipmen.  Before  commencing  this  work,  the  specific  performance  metrics 
that  would  be  analyzed  needed  to  be  determined.  The  basis  for  making  this  decision  was 
the  results  of  the  survey  which,  as  detailed  above,  was  developed  based  on  the  results  of 
stakeholder  interviews  and  information  garnered  firom  the  literature  review.  Specifically, 
the  first  question  in  the  survey  asked  Company  Officers  to  list  the  metrics  that  they  used 
to  track  the  performance  of  their  midshipmen.  Any  metric  selected  by  more  than  half  of 
the  Company  Officers  surveyed  was  included  on  the  list  for  analysis.  The  list  is  shown  in 
Table  IV-2.  Each  of  the  selected  metrics  will  be  discussed  in  turn. 
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Metric 

Response  Ratio 

PRT  scores 

88% 

Overall  QPR 

92% 

Changes  in  QPR 

76% 

Class  absences 

72% 

Number  of  midshipmen  on  academic  probation 

80% 

Number  of  D’s/F’s  in  military  performance 

72% 

Pro  quiz  grades 

64% 

Appearance/personnel  inspection  results 

68% 

Room  inspection  results 

56% 

Number  of  academic  board  cases 

60% 

Number/severity  of  conduct  offenses 

88% 

Company  drill  grades 

56% 

Table  IV-2:  Metrics  Selected  for  Analysis 


Most  of  the  selected  metrics  are  applicable  to  both  tracking  the  performance  of 
individuals  and  the  performance  of  the  company  as  a  whole.  Thus,  methods  for 
extracting  information  pertinent  to  tracking  both  individual  and  company  performance 
were  developed  as  appropriate.  The  currently-available  MIDS  modules,  particularly  the 
Company  Officer  System  module,  is  quite  comprehensive  when  it  comes  to  individuals, 
providing  nearly  all  the  information  necessary  to  track  a  single  person’s  performance. 
Company-aggregate  information  is  somewhat  harder  to  come  by.  A  few  company-wide 
metrics  in  MIDS  are  foirad  in  the  Company  Officer  -  Summary  Infomiation  Module. 

Where  information  for  a  particular  metric  was  available  in  currently-existing 
MIDS  modules,  the  assessment  consisted  of  locating  that  information  and  detailing  how 
it  could  be  accessed.  This  was  how  the  reports  for  most  individual  performance 
measurements  and  few  company-wide  measurements  were  created.  In  cases  where 
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adequate  information  could  not  be  extracted  from  existing  MIDS  modules,  the 
Webintelligence  software  was  employed  to  access  and  display  the  needed  information. 
Most  reports  for  company-wide  metrics  were  developed  in  this  manner.  When  possible, 
charts  and  graphs  are  used  to  enhance  the  visibility  of  trends  in  the  data.  Since  all 
modules  of  MIDS  are  text  and  tables  only,  Webintelligence  queries  were  required  to 
produce  any  sort  of  graphics.  In  two  cases,  the  structure  of  the  data  in  MIDS  was  such 
that  neither  currently-existing  modules  nor  Webintelligence  provided  a  sufficient  method 
for  viewing  the  desired  information.  In  these  cases,  Webintelligence  was  used  to  acquire 
the  data,  which  was  exported  to  Microsoft  Excel  for  further  manipulation  and  display. 

Finally,  for  four  of  the  twelve  metrics  selected,  no  data  exists  in  the  MODS 
database  that  is  pertinent  to  that  metric.  Because  of  this,  it  was  impossible  to  create  any 
performance  measurement  methods  for  those  metrics.  The  metrics  so  affected  were  pro 
quiz  grades,  appearance/personnel  inspection  results,  room  inspection  results,  and 
company  drill  grades.  If,  at  some  point  in  the  future,  information  on  these  is  added  to  the 
MIDS  database,  procedures  similar  to  those  outlined  below  could  be  used  to  track  and 
display  individual  and  company  performance  in  these  areas.  The  remaining  eight  metrics 
are  dealt  with  in  detail  below.  Step-by-step  procedures  on  how  to  access  and  produce  the 
reports  discussed  below  are  provided  in  Appendix  B. 

1.  PRT  Scores 

Current  and  previous  PRT  scores  for  individual  midshipmen  were  easily  accessed 
through  the  MIDS  Company  Officer  System  module.  A  table  provided  in  each 
midshipman’s  report  lists  his  or  her  scores  for  each  of  the  component  tests  and  the  PRT 
as  a  whole  for  every  instance  they  have  taken  the  test  (Figure  rV-2).  This  table  allows  the 
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Company  Officer  to  rapidly  assess  both  the  current  PRT  status  of  an  individual  as  well  as 
showing  any  trends  in  that  person’s  PRT  performance  over  time. 
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Figure  IV-2:  Individual  PRT  Results  in  MIPS 

No  MIDS  module  provides  PRT  scores  for  the  company  as  an  aggregate,  so  a 
Webintelligence  query  was  created  to  extract  the  average  score  for  each  company  in  the 
battalion  for  each  semester  over  the  course  of  several  years.  (The  user  can  set  the  period 
covered  by  the  report.)  The  resulting  table  of  scores  was  imported  into  Microsoft  Excel 
and  converted  into  a  line  chart  showing  each  of  the  scores  for  each  company  in  the 
battalion,  by  semester,  over  the  reporting  period.  Excel  was  chosen  to  produce  this  graph 
because  of  its  ability  to  dynamically  scale  the  y-axis  of  the  plot  to  best  show  the  range  of 
scores  covered  (Figure  IV-3).  Webintelligence  was  capable  of  producing  a  plot  as  well, 
but  since  the  scale  is  fixed  (in  this  case,  from  zero  to  100)  and  the  variation  in  average 
scores  between  compames  and  over  time  is  relatively  small,  all  the  lines  were  clustered  at 
the  top  of  the  axis  and  different  lines  were  indistinguishable. 
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Figure  rV-3:  Company  PRT  Scores  Within  the  Battalion 


2.  Overall  QPR  and  Changes  in  QPR 

As  with  PRT  scores,  individual  academic  grades  (and  QPR’s)  were  available  as 
part  of  the  Company  Officer  System  module  report  on  the  individual  midshipman.  In 
order  to  better  visualize  an  individual’s  trends  in  QPR,  a  graph  was  created  using 
Webintelligence  that  shows,  for  a  given  midshipman,  his  or  her  semester  QPR  and 
cumulative  QPR  (through  the  corresponding  semester)  for  each  semester  of  attendance  at 
the  Academy  (Figure  IV-4),  The  cumulative  QPR  line  shows  the  individual’s  general 
trend  in  academics,  while  the  semester  QPR  line  highlights  uncharacteristically  good  and 
bad  semesters. 
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Figure  rV-4:  Individual  Semester  and  Cumulative  QPR  Over  Time 

In  order  to  track  company-wide  academic  performance  using  QPR’s,  two  separate 
reports  “were  created.  The  first  report,  a  bar  graph  created  using  Webintelligence,  is  a  line 
graph  showing  the  average  semester  QPR  for  each  class  year  group  of  midshipmen  in  the 
company  (Figiue  rV-5).  This  report  allows  the  Company  Officer  to  compare  the 
performance  of  different  portions  of  his  company  over  several  years. 
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Figure  IV-5:  Company  QPR  Trends,  by  Class  Year 


The  second  company-wide  report  is  a  comparison  of  the  average  semester  QPR 
for  each  company  in  the  battalion  over  a  user-selectable  time  period  (Figure  IV-6).  This 
report  is  an  Excel-generated  line  graph  created  from  data  extracted  using  a 
Webintelligence  query.  As  with  the  PRT  scores,  this  combination  of  Webintelligence 
and  Excel  was  used  because  of  the  axis-scaling  limitations  of  Webintelligence  and  the 
small  variation  in  scores  relative  to  the  size  of  the  scale.  This  plot  allows  the  Company 
Officer  to  track  how  his  company  is  doing  over  the  period  of  the  report  as  compared  with 
other  companies  in  the  battalion  over  the  same  period. 


75 


1-FALL  I  2-SPRING 
1998 

Figure  IV-6:  Company  Semester  QPR  Within  the  Battalion 
3.  Class  Absences 

Absences  were  a  more  comphcated  metric  than  the  previous  two.  In  addition  to 
the  individual  and  company  levels  of  analysis,  there  were  distinctions  to  be  made 
between  total  absences,  including  those  for  legitimate  reasons,  and  unexcused  absences. 
The  latter  is  typically  a  responsibihty  and  conduct-related  matter,  while  the  former  may 
serve  to  explain  changes  in  academic  performance.  To  try  to  capture  some  of  this 
complexity,  six  reports  were  employed.  Two  of  these  are  based  on  individuals  within  the 
company,  one  compares  class  year  groups  within  the  company,  and  three  compare  the 
companies  within  the  battalion. 


The  most  comprehensive  information  on  individual  absences  within  the  company 
is  provided  through  the  Company  Officer  -  Summary  Information  MIDS  module. 
Entering  the  company  in  question  and  selecting  “Absences:  All”  for  the  type  of  report 
(Figure  rV-7)  generates  a  matrix  list  of  all  midshipmen  in  the  company  and  their  total 
number  of  absences  for  the  current  semester,  broken  down  by  the  various  types  of 
excusal  reasons.  With  this  matrix,  the  Company  Officer  has  access  to  detailed 
information  on  class  attendance  for  all  midshipmen  in  the  company.  There  are  three 
significant  drawbacks  to  this  report.  First,  the  report  is  sorted  alphabetically,  which 
makes  it  difficult  to  spot  problem  areas.  Second,  it  only  provides  cumulative  data  for  the 
current  semester,  precluding  analysis  of  either  short-term  phenomena  or  multi-semester 
trends  for  particular  individuals.  Finally,  there  is  no  aggregation  of  company-wide 
information  in  the  report. 


Company  Officer  -  Summary  Information 

Company:||o-| 

C  Absences:  All 

Athletic  Status 

C  Leave  Status  ^  Probation  Status 

C  Absences:  >=  |10 

C  Conduct  Status 

C  Merit  List  Status  PRT  Status 

C  Absences:  Top  10 

('  EGA  Status 

C  Movement  Orders  and  Excusals  C  Striper  Status 

C  UA  Tardy:  Top  10 

r*  Dovmload  !Eile 

'T  General  Information 

C  QPRs  and  Standings 

Figure  IV-7:  Company  Officer  -  Sxmimary  Information  Module  Input  Screen 


Another  portion  of  the  Company  Officer  -  Summary  Information  MIDS  module 
addresses  the  first  problem.  By  selecting  the  “Absences;  Top  10”  checkbox,  the 
Company  Officer  can  generate  a  list  of  the  top  ten  midshipmen  with  the  most  total 
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absences  in  the  company,  sorted  by  the  number  of  absences.  This  allows  the  Company 
Officer  to  immediately  spot  midshipmen  with  potential  excessive  absence  problems. 
This  report,  though,  also  is  based  on  cumulative  semester  data,  and  thus  suffers  firom  the 
same  scope-of-analysis  problems  as  the  “Absences:  All”  report. 

The  remaining  four  reports  deal  with  company-wide  data.  They  also  address  the 
scope-of-analysis  problem  by  providing  user-selectable  time  periods  that  can  be  as  little 
as  a  single  day  or  as  much  as  several  semesters.  The  simplest  of  these  reports  is  a 
Webintelligence-created  bar  graph  showing  the  total  number  of  unexcused  absences  in 
the  company  over  the  selected  period  (Figme  IV-8).  The  example  shown  is  for  a  two 


Company  TJA^s 


Number  of  Absent/Tardy/Left  Early 


03-Apr-2001  04-Apr-2001  09-Apr-2001  11 -Apr-2001 
Absenteeism  Date 


Figure  IV-8:  Company  Unexcused  Absences 
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iveek  period.  Note  that  only  days  were  there  was  one  or  more  unexcused  absences  are 
included  on  the  x-axis.  This  is  a  limitation  of  the  Webintelligence  software. 

A  more  complicated  and  informative  version  of  this  report  compares  the  number 
Df  unexcused  absences  for  all  the  companies  in  a  battalion  over  a  given  period  of  time 
'Figure  IV-9). 

The  remaining  two  reports  measure  the  total  number  of  absences,  excused  or  not. 


Company  UA's  vs.  Battalion 


Number  of  AbsentH'ardy/Left  Early 


Absenteeism  Date 


Figure  IV-9:  Company  Unexcused  Absences  Within  the  Battalion 

The  first  of  these  (Figure  IV-10)  shows  all  of  the  absences  for  the  company,  broken  down 
by  class  year  group.  The  other  (Figure  IV- 11)  shows  all  of  the  company’s  absences 
compared  with  the  other  compames  in  the  battalion.  One  important  use  of  this  report  is 
to  determine  if  spikes  in  company  absences  are  an  isolated  incident,  or  are  related  to 
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some  battalion-  Or  brigade-wide  event  (such  as  an  academic  conference  or  major  athletic 
event).  As  with  the  two  previous  reports,  these  are  both  Webintelligence  bar  charts  that 
can  cover  any  time  span  desired  by  the  Company  Officer. 


Figure  IV-10:  Company  Absences  by  Class  Year  Group 
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Company  Attendance  vs.  Battalion 


Number  of  Absent/Tardy/Left  Early 


Absenteeism  Date 


Figure  IV- 1 1 :  Company  Absences  Within  the  Battalion 


4.  Academic  Probation  and  Academic  Boards 

In  addition  to  the  QPR  information  discussed  above,  some  secondary  metrics  are 
available  to  the  Company  Officer  in  assessing  the  company’s  academic  performance. 
According  to  the  interview  and  survey  results,  the  most  closely  watched  of  these 
secondary  metrics  is  the  number  of  midshipmen  on  academic  probation  at  any  given  time. 
To  assist  the  Company  Officer  in  tracking  this  metric,  two  reports  were  employed.  The 
first  report  was  a  list  of  those  individuals  in  the  company  currently  on  academic 
probation.  This  list  is  part  of  the  Company  Officer  -  Summary  Information  MIDS 
module,  and  can  be  obtained  by  selecting  the  “Probation  Status”  checkbox  when 
accessing  the  module  (Figure  IV-7).  The  second  report,  a  Webintelligence  line  graph, 
compares  the  number  of  midshipmen  per  semester  on  academic  probation  for  the 
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companies  in  the  battalion  (Figure  IV-12).  Alternatively,  this  same  report  can  be 
manipulated  to  just  show  statistics  for  a  single  company  if  the  Company  Officer  so 
desires.  (See  Appendix  B  for  details.) 

A  related  metric  is  the  number  of  midshipmen  that  are  sent  to  academic  review 


Figure  IV-12:  Midshipmen  on  Academic  Probation  for  Each  Company  in  the  Battalion 

boards  each  semester.  This  can  be  indicative  of  the  company’s  academic  performance  as 
well  as,  when  compared  with  the  previous  report,  how  well  the  company  does  at  assisting 
those  in  academic  trouble  during  the  semester,  preventing  them  firom  having  to  go  to  an 
academic  board.  The  report  used  here  is  a  Webintelligence  bar  chart  displaying  die  total 
number  of  academic  board  cases  for  each  company  in  the  battalion  (Figure  IV- 13). 
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Figure  IV-13:  Number  of  Academic  Board  Cases  Per  Company  in  the  Battalion 


The  previous  three  reports  in  this  section  focused  on  secondary  metrics  that 
identified  midshipmen  on  the  bottom  end  of  the  academic  performance  spectrum.  It  is 
also  important  for  the  Company  Officer  to  identify  and  track  the  performance  of  those 
that  are  at  the  top  end  of  this  spectrum.  The  Company  Officer  -  Summary  Information 
MIDS  module  provides  a  method  for  easily  identifying  those  individuals.  Those 
midshipmen  that  excel  in  academics,  as  well  as  meeting  different  physical,  professional, 
and  conduct  standards,  are  placed  onto  one  of  several  merit  lists  and  are  awarded 
corresponding  privileges.  By  selecting  the  “Merit  List  Status”  checkbox,  this  module  can 
be  used  to  generate  a  list  of  midshipmen  in  the  company  that  qualify  for  one  or  more  of 
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the  merit  lists  for  the  current  semester.  As  with  all  reports  generated  by  this  module, 
however,  there  is  no  provision  for  tracking  this  information  across  multiple  semesters. 

5.  Military  Performance  Deficiencies 

The  next  metric  to  be  analyzed  was  midshipmen  with  failing  marks  in  military 
performance.  Again,  the  Company  Officer  -  Summary  Information  MEDS  module  can  be 
used  to  generate  a  list  of  those  midshipmen  in  a  company  on  military  performance 
probation.  In  this  case,  the  appropriate  checkbox  is  “Probation  Status”.  In  order  to  track 
military  performance  failures  against  the  other  companies  in  the  battalion,  a 
Webintelligence  bar  chart  was  created  that  shows  the  number  of  midshipmen  receiving  a 
D  or  F  grade  in  military  performance  for  the  semester  in  each  company,  for  a  semester 
specified  by  the  user  (Figure  rV-14).  An  important  characteristic  of  diis  report  is  that, 

Number  of  Midshipmen  | 

5 

4 

3 

2 

1 

0 

A  B  C 

Mid's  Company  Number 

Figure  IV-14:  Midshipmen  With  Failing  Mihtary  Performance  Grades  in 
Each  Company  Within  the  Battalion  for  a  Given  Semester 
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unlike  most  of  the  other  Webintelligence-generated  graphs  developed  in  this  project,  it 
only  displays  one  semester.  This  is  a  limitation  imposed  by  the  architecture  of  the  MODS 
database  that  prevents  more  than  one  semester’s  worth  of  military  performance  grades 
from  being  accessed  in  the  same  query.  Thus,  to  get  an  idea  of  trends  for  this  metric,  the 
user  would  need  to  run  the  query  once  for  each  semester  in  question,  print  or  save  each 
output,  and  visually  compare  the  graphs. 

6.  Conduct  Offences 

The  final  metric  considered  in  this  research  was  the  number  and  severity  of 
conduct  offences.  Records  for  individuals  are  most  easily  accessed  via  the  Company 
Officer  System  MIDS  module,  where  a  portion  of  each  midshipmen’s  report  details  their 
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Figure  rV-l 5:  Individual  Conduct  Records  in  MIDS 
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individual  conduct  history  (Figure  IV-15).  As  with  academic  and  military  performance 
probation,  a  list  of  midshipmen  in  the  company  on  conduct  probation  can  be  generated  by 
using  the  “Probation  Status”  checkbox  in  the  Company  Officer  —  Summary  Information 
MIDS  module. 

The  second  report  employed  to  track  this  metric  is  a  measure  of  the  number  of 
midshipmen  committing  conduct  offenses  in  the  company  per  semester.  This  is  broken 
down,  using  a  Webintelligence  bar  chart,  by  major  and  minor  offenses,  allowing  the 
Company  Officer  to  track  the  enforcement  level  of  (relatively  constantly  occurring) 


Figure  rV-16:  Midshipmen  Committing  Conduct  Offenses  Per  Semester 


linor  infractions  as  well  as  the  incident  rate  of  major  conduct  offenses  over  the  course  of 
;veral  semesters  (Figure  IV-16). 

The  final  report  in  this  section  was  created  to  compare  the  company’s  conduct 
;cord  for  major  infi'actions  with  the  rest  of  the  battalion.  To  this  end,  a  Webintelligence 
ar  chart  was  created  that  plots  the  total  number  of  midshipmen  in  each  company  that 
ommitted  a  major  conduct  offense  each  semester  (Figure  IV-17).  As  is  the  case  with 
lost  of  these  reports,  the  period  covered  by  the  report  is  selectable  by  the  user. 


Figure  IV-17:  Midshipmen  Per  Company  Committing  Major  Conduct 
Offenses  Each  Semester  Within  the  Battalion 


CHAPTER  SUMMARY 

MCDS  has  evolved  into  an  extremely  useful  data  storage  and  retrieval  system  for 
se  by  personnel  at  the  Academy.  However,  Company  Officers  can  increase  the  system’s 
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usefulness  by  using  the  system  as  a  perfonnance  measurement  tool  in  the  development  of 
midshipmen.  For  this  to  occur,  Company  Officers  must  play  an  integral  part  in  the 
identification  of  metrics  to  be  tracked  and  the  designation  of  standards  to  be  followed  for 
each  of  them.  This  participation  will  ensure  that  all  Company  Officers  and  midshipmen 
xmderstand  current  and  future  perfonnance  goals  and  that  a  fair  and  clearly  understood 
environment  exists  in  regards  to  performance  grades  and  the  Naval  Academy.  The 
survey  was  designed  to  capture  the  opinions  Company  Officers  have  about  MIDS.  This 
chapter  discussed  those  opinions  and  subsequently  used  the  responses  to  show  techniques 
where  MIDS  could  be  used  to  measure  the  performance  metrics  important  to  the 
Company  Officers.  The  data  collected  can  now  be  used  to  give  recommendations  for 
future  development  of  the  MEDS  system.  The  following  chapter  offers  conclusions  about 
the  current  use  of  MEDS  by  Company  Officers  and  presents  recommendation  for  future 
research. 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


A.  SUMMARY  AND  CONCLUSIONS 

This  research  examined  use  of  the  currently  deployed  Midshipmen  Information 
System  (MIDS)  as  a  performance  measurement  tool.  MIPS  is  a  data  warehouse  system 
designed  to  store  performance-based  information  for  use  by  the  faculty,  staff  and 
midshipmen  at  the  United  States  Naval  Academy.  MODS  came  online  in  1999  and  is 
intended  for  use  by  Company  Officers  as  a  performance  measurement  tool  in  the 
development  of  midshipmen. 

Research  conducted  for  this  thesis  involved  an  in-depth  review  of  current 
performance  measurement  theories  that  exist  in  both  the  public  and  private  sector. 
Previous  research  in  the  area  of  performance  measurement  at  the  United  States  Naval 
Academy  was  also  examined.  To  assess  overall  use  of  MIDS,  group  interviews  of 
faculty,  staff,  and  midshipmen  were  conducted  by  Leadership  Education  and 
Development  (LEAD)  students.  To  specifically  gain  Company  Officer  usage  data,  an 
online  survey  was  given  to  each  officer  currently  holding  a  Company  Officer  billet. 
Twenty-five  Company  Officers  participated  in  the  survey.  These  Company  Officers 
provided  valuable  insight  into  their  understanding  of  MIDS,  performance  measurement, 
and  what  metrics  they  personally  feel  are  important  to  the  development  of  midshipmen. 

The  interview  responses  showed  several  widely  held  opinions  with  respect  to 
MIDS  and  performance  measurement  in  general.  First,  very  little  training  exists  to  assist 
end  users  in  ntiliVing  MIDS  to  its  fullest  capability.  Secondly,  the  academy’s  mission  “to 
develop  midshipmen  morally,  mentally,  and  physically”  (USNA,  2001a,  p.  1)  does  not 
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set  standards  as  to  how  this  midshipmen  development  should  be  measured.  Very  little 
guidance  is  given  to  Company  Officers  on  how  to  translate  this  mission  into  metrics  and 
standards  of  performance  that  are  appropriate  for  measuring  the  desired  development. 

Responses  to  the  Company  Officer  online  survey  were  diverse,  though  many 
trends  could  be  seen  in  opinions  about  ME)S.  First,  Company  Officers  felt  that  very 
little  training  and  documentation  exists  to  assist  them  with  their  use  of  MIDS  and  that 
more  training  on  tbe  system  would  help  them  conduct  their  job  more  efficiently.  Metrics 
used  to  assess  the  performance  of  their  midshipmen  varied  widely,  although  a  few 
metrics  seemed  important  to  most  of  the  Company  Officers.  These  metrics  included 
number  of  midshipmen  on  academic  probation  (“UNSAT”)  in  company,  overall  QPR, 
PRT  results,  and  number  and  severity  of  conduct  offenses.  Very  few  Company  Officers 
utilize  visual  aids  such  as  graphs  and  charts  to  show  trends  to  their  company  or  their 
superiors  regarding  midshipmen’s  performance  over  time.  In  general,  no  consistent 
practices  exist  between  Company  Officers  in  relation  to  their  use  of  MIDS  as  a 
performance  measurement  tool. 

The  final  portion  of  the  project  consisted  of  a  system  capability  analysis  of  MIDS 
and  the  Webintelligence  ad-hoc  query  software.  Eight  different  performance  metrics 
were  selected  for  this  analysis.  These  were  chosen  because  they  were  considered 
important  by  a  majority  of  the  Company  Officers  surveyed  and  data  relevant  to  them  was 
present  in  the  MIDS  database.  Four  other  metrics  were  considered  important  by  the 
Company  Officers,  but  since  no  data  existed  in  MIDS  to  measure  these,  they  were  not 
included.  Currently  existing  MIDS  modules  were  investigated  to  determine  their  utihty 
in  tracking  these  metrics.  If  the  existing  modules  were  not  adequate  for  tracking  these 
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metrics,  Webintelligence  was  used  to  generate  appropriate  reports.  In  some  cases,  it  was 
also  necessary  to  employ  external  software  (Microsoft  Excel)  to  aid  in  displaying  the 
information  in  a  form  useful  to  the  Company  Officer.  Standardized  procedures  were 
created  for  Company  Officers  to  access  the  various  reports  examined  or  created  in  this 
phase.  These  reports  and  procedures  should  be  useful  to  Company  Officers  in  tracking 
the  performance  of  their  midshipmen,  and  can  serve  as  templates  for  the  creation  of 
further  reports  in  the  future. 

Overall,  MIDS  is  an  excellent  system  for  monitoring  the  performance  of 
midshipmen.  It  contains  a  wealth  of  information  on  many  performance-related  issues  and 
is  fairly  simple  to  use.  Webintelligence  is  a  valuable  supplement  to  MIDS.  Its 
customizable  data  access  and  visualization  capabilities  are  quite  useful.  On  the  other 
hand,  several  technical  limitations  to  MIDS  and  Webintelligence  were  discovered  during 
the  system  capability  analysis.  Improving  one  or  more  of  these  areas  would  significantly 
increase  the  usefulness  of  MIDS  to  the  Company  Officer. 

First,  MIDS  is  very  good  at  presenting  individual  data,  but  its  capacity  for 
displaying  company-aggregate  data  is  limited.  Some  portions  of  the  Company  Officer  — 
Summary  Information  module  provide  company-wide  reports,  but  more  would  be  useful. 

Another  limitation  of  MIDS  is  that  its  capabilities  for  displaying  information  over 
time  varies  widely.  In  some  cases,  such  as  individual  academics,  conduct,  and  PRT 
grades,  long-term  information  is  readily  available  and  easy  to  interpret.  In  other  cases, 
such  as  absences,  probations,  and  merit  lists,  no  method  exists  for  viewing  any  historical 
information. 
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Finally,  MIDS  is  a  text-only  system,  and  displays  most  of  its  information  in 
tables.  While  this  is  appropriate  for  getting  exact  values  of  a  particular  piece  of 
graphical  presentation  (bar  or  line  charts,  for  example)  would  be  more  effective  for 
identification  of  trends.  Webintelligence  has  graphical  capabilities,  but  the  query 
construction  interface  is  somewhat  complicated,  particularly  for  the  casual  user. 

B.  ACTION  RECOMMENDATIONS 

The  Information  Technology  Systems  Division  (ITSD)  at  the  United  States  Naval 
Academy  is  responsible  for  the  MIDS  system.  However,  its  personnel  cannot  be 
responsible  for  all  aspects  of  the  system.  Performance  measurement  is  always  changing, 
with  new  ideas  being  generated  almost  daily.  Therefore,  it  is  the  responsibility  of  each 
user  to  search  for  areas  to  improve  use  of  MIDS.  This  thesis  provides  four  areas  to  be 
examined  for  improvement  of  MIDS,  improvement  of  Company  Officers’  use  of  die 
system,  and  use  of  that  system  in  the  development  of  midshipmen. 

1.  Training  for  Company  Officers 

Develop  a  training  curriculum  that  could  be  included  in  the  LEAD  program’s 
Performance  Measurement  class.  This  training  would  assist  with  the  indoctrination  of 
Company  Officers  into  their  roles  and  get  them  competent  with  the  use  of  MIDS  before 
their  move  into  Bancroft  Hall.  This  training  would  also  assist  Company  Officers  by 
reducing  the  amount  of  time  they  must  spend  on  the  computer  obtaining  information. 
Working  more  efficiently  would  allow  for  more  time  to  be  spent  in  a  mentor  position 
with  the  midshipmen.  Understanding  the  system  better  would  allow  the  Company 
Officer  to  extract  performance  trends  that  they  could  show  to  individual  midshipmen  and 
their  entire  companies.  This  training  would  also  result  in  the  Company  Officer  having  a 
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more  thorough  understanding  of  each  midshipmen  and  their  company’s  performance 
because  of  their  ability  to  see  trends. 

2.  Set  Institutional  Standards 

In  order  to  ensure  that  midshipmen  receive  fair  and  consistent  treatment  during 
their  development  into  naval  officers,  standards  must  be  set  concerning  the  way  their 
performance  is  measured.  The  institutional  leadership  should  indicate  what  metrics  are 
important  to  track,  and  each  company  officer  should  follow  those  guidelines  to  ensure 
consistent  results.  All  stakeholders  should  be  involved  in  the  process  of  generating  these 
guidelines  so  that  all  areas  of  midshipmen  development  are  included  and  that 
performance  measurement  at  the  Academy  is  up  to  date  with  current  performance 
measurement  models. 

3.  Add  Any  Needed  Data  to  MIDS 

One  limitation  of  the  system  capability  analysis  was  that,  for  four  of  the  twelve 
metrics  considered  important  by  the  Company  Officers,  no  relevant  data  was  available  in 
the  MIDS  database.  Once  a  list  of  standard  performance  metrics  are  decided  upon,  this 
list  needs  to  be  checked  against  the  data  available  in  MIDS.  If  there  are  any  metrics  for 
which  no  data  is  available,  new  data  tables  and  modules  for  inputting  and  viewing  this 
new  data  need  to  be  created. 

4.  Develop  Standard  Procedures  for  Tracking  Metrics 

Once  the  important  metrics  have  been  determined  and  data  is  available,  the  next 

step  is  to  develop  and  implement  standard  procedures  for  tracking  and  reporting  those 
metrics.  Appendix  B  is  one  example  of  what  these  procediures  might  look  like.  Ideally, 
these  procedures  should  provide  the  Company  Officer  with  a  systematic  process  to 
generate  reports  pertinent  to  each  selected  metric.  This,  combined  with  Brigade-wide 
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perfoimance  standards,  would  go  a  long  way  to  reducing  the  variability  in  performance 
measurement  between  companies. 

C.  AREAS  FOR  FUTURE  RESEARCH 

Due  to  the  ever-changing  ideas  in  performance  measurement  and  the  needs  and 
expectations  of  the  United  States  Naval  Academy  and  the  fleet,  a  yearly  assessment  of 
performance  measurement  at  the  Academy  is  required.  This  assessment  could  be  done  by 
a  LEAD  student  and  would  satisfy  three  key  areas.  The  first  would  be  fulfilling  the  thesis 
requirement  of  a  LEAD  program  student.  The  second  would  be  to  reduce  the  workload 
placed  upon  the  ITSD.  The  last  would  be  to  continually  have  an  up  to  date  performance 
measurement  system  available  for  the  Academy  that  complies  with  the  Government 
Performance  and  Results  Act  (GPRA). 

A  second  possible  follow-up  would  be  to  conduct  a  comparative  study  of 
performance  measurement  between  USNA  and  other  large  federal  institutions.  Obvious 
choices  for  comparisons  would  be  the  Air  Force  Academy,  Coast  Guard  Academy  and 
Military  Academy.  Other  portions  of  the  military  or  civilian  government  agencies  may 
also  be  mstructive  to  study.  This  type  of  study  could  include  the  interplay  between 
institutional  strategy  and  performance  measurerrient,  metrics  considered  important, 
methods  for  setting  standards,  and  how  metrics  are  tracked  and  reported. 

Another  possible  study  would  be  an  assessment  of  customer  satisfaction  with  the 
Academy’s  product,  its  graduates.  In  this  case,  the  Academy’s  customers  are  the 
commanding  officers  in  the  fleet.  The  goal  of  this  type  of  study  would  be  to  determine 
the  quality  of  graduates,  as  seen  by  diese  fleet  commanding  officers,  and  compare  this 
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with  the  perfonnance  measurement  standards  and  metrics  at  the  Academy.  In  addition, 
this  would  provide  a  “reality  check”  on  the  performance  goals  of  the  Academy  by 
comparing  what  attributes  fleet  commanding  officers  want  to  see  in  new  officers  with  the 
attributes  that  the  Academy  is  striving  to  develop. 
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APPENDIX  A:  SURVEY  AND  RESULTS 


1. 

Which  of  the  following  metrics  do  you  use  to  track  the  performance  of 
individual  midshipmen  or  your  company  as  a  whole?  (check  all  that 
apply) 

ilumber  of 
flespons«$ 

Response 

Ratio 

PRT  scores 

[ _ 

22 

88 

PE  grades 

mmmm 

7 

28% 

Overall  QPR 

1 

24 

98% 

Changes  in  QPR 

[ _ 

18 

72% 

19 

78% 

Study  hours 

3 

12% 

El  hours ;  flU 

3 

12% 

20 

80% 

Performance  ! 

18 

72% 

4 

16% 

Pro  Quiz  grades 

16 

64% 

Appearance/personnel  inspection 
results 

17 

68% 

Room  inspection  results 

14 

56% 

Number  of  academic  board  cases 

15 

80% 

Number  honor  cases  | 

12 

48% 

22 

88% 

onenses  | 

Number  of  weight  category  5/6  j 
cases i 

10 

40% 

Attendance  at  company  functions  | 

9 

36% 

Community  involvement  | 

5 

20% 

SIR  chits  per  midshipman  ' 

3 

12% 

Company  intermurat  performance  : 

11 

1 _ 

44% 

Company  drill  grades 

14 

56% 

— 

5 

20% 

Which  of  the  following  metncs  do  you  use  to  track  the  performance  of  individual  midshipmen 
or  your  company  as  a  whole?  (check  all  that  apply) 

1  Response  j 

1 

conduct  history,  conduct  action  pending,  movement  orders 

2 

Chain  of  Command  input 

3 

Number  of  Mids  on  ment  lists  (Deans.  Supts.  Dants);  Company  QPR  (Semester  and  Cum);  Company  QPR 
difference  (Semester) 

4 

PE  deficiencies  (swimming,  PE  failures,  etc.) 

5 

Time  1  spend  talking  eeith  professors 
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-2/; 

Do  you  track  chan; 
mishipmen  or  your 

jes  in  performance  *over  time*  for  either  individual 
whole  company? 

Niunberof 

Responses 

Response 

Ratio 

No 

24 

1 

96% 

4% 

Total 

25 

100% 

3. 

Do  you  use  any  visual  representations  (graphs,  etc.)  of  performance 
for  your  use,  your  company's,  or  your  Battalion  Officer's? 

Nliiiiiberof 

Responses 

Response 

Ratio 

Yes 

10 

40% 

NO 

Total 

15 

25 

Which  of  the  following  most  closely  describes  how  often  you  use  the 
MIDS  system? 

Numbef  of 
Responses 

Response 

R«io 

Onoe  a  week  or  less 

0 

0% 

Three  times  a  week 

0 

0% 

Once  a  day 

• 

2 

I 

8% 

Twice  a  day 

Q 

38% 

Four  or  more  times  a  day 

14 

56% 

Total 

25 

100% 
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E 


What  do  you  use  MIDS  for?  (Please  rank  in  order  of  importance.) 


Percejrtage  mdscstes  tot&t  respondent  ratio 
end  parenthesis  rndkete  actuat  number. 

.  ^freqijent 

'  iW/A  .  ' 

1.  Track  th«  performance  of  individual 

4% 

midshipmen 

(1) 

2,  Track  the  performance  of  your  company 

0% 

20% 

32% 

28% 

12% 

8% 

as  a  whole 

CO) 

(5) 

0) 

(7) 

C3) 

C2) 

3.  Produce  required  reports  for  your 

36% 

32% 

4% 

Battalion  Officer 

C9) 

(8) 

Cl) 

4.  Gather  information  for  conducVhonor/ 

24%  . 

44% 

8% 

24% 

0% 

0% 

academic/performance  boards 

(6) 

(11) 

G2) 

(P) 

0% 

0% 

5.  Enter  midshipmen  performance  grades 

CO) 

CO) 

0% 

4% 

8% 

6.  Other  (please  state  below) 

CO) 

(1) 

C2) 

r'' 

Other 

1  Response 

1 

Find  personal  Information  (e.g,  major,  home  of  record,  sports  activities,  contact  phene  numbers  for  leave  periods 

2 

Mostly  checking  individual  records  when  they  submit  chits,  checking  absences  and  cross  referencing  them  with 

MO's.  etc. 

3 

approving  excusals/movement  orders,  finding  out  free  periods  so  i  can  talk  to  one  of  my  mids... 

4 

Initiate.  Review,  and  Approve  Movement  Orders  and  Excusals 

5 

MO/Excusal  approval;  MiscINFO  re:  MIDN 

6 

Gather  information  forthings  otherthan  conduct/academlc^etc.  boards 

7 

Counseling,  spot  checking  and  MOs 

\  -- 

99 


.^-1.  .. 

:•  :•  =7.5 

Which  module  of  MIDS  do  you  use  the  most? 

Niunberof 

ResporMs 

Ratio 

Company  Officer  Page 

L _ l^B 

22 

88% 

Company  Officer  -  Summary 
information 

1 

4% 

Matrices  -  Query  Current 
Midshipmen 

0 

0% 

Weekend  Eligibility 

0 

0% 

Absences  j 

0 

0% 

Other.  Please  Specify 

8% 

Total 

25 

100% 

Do  you  .use  the  ad-hoc  query  system  (Webintelligence)? 

Number  of 
Responses 

Response 

Ratio 

23 

96% 

No  , 

1 

4% 

Total 

24 

100% 

- r - - -  - - - — . . — — - 

^  If  so,  how  many  times  per  week? 

Numbesrof 

Responses 

Response 

Ratio 

Once  : 

9 

39% 

Three  times : 

12 

52% 

Five  times ; 

1 

4% 

Seven  times 

0 

0% 

Nine  or  more  times 

1 

4% 

Total 

23 

!  100% 

10.  How  well  does  MIDS  functionality  meet  your  needs? 

Numberof 

Resp^uises 

RespiHise 

Ratio 

Not  at  all  1 

0 

0% 

A  little  2 

0 

0% 

Somewhat  3 

3 

12% 

21 

84% 

Completely  5 

1 

4% 

Total 

25 

100% 

100 


p'  'im 

What  new  functions  or  data  would  you  like  to  see  from  MIDS? 

# 

;  Response  j 

1 

Platoon  Cdi*s  notebook 

2 

Ad  Hoc  queries  are  not  always  accurate  depending  on  when  and  where  the  data  is  drawn.  This  o^en  leads  to 
confusion. 

1  would  like  to  see  more  functions  that  enable  me  to  look  at  my  company,  as  a  whole,  overtime.  Company  Officer. 
Summary  Information  Is  only  marginally  useful. 

3 

Ad  hoc  query  and  other  functions  don't  print  well.  Approve/deny  all  for  Movement  Orders  is  useless  (need  an 
approve/deny  eligible) 

4 

Oraphs  for  trends  company  and  individual  level. 

5 

Muster  results,  inspection  results 

6 

More  items  tailored  for  the  companyvice  the  academic  departments. 

7 

1  would  like  to  see  an  automatic  cross  reference  between  absences  and  MO's.  If  a  mid  comes  up  absent,  you  have 
to  go  into  a  completely  different  module  to  see  if  he  was  really  on  an  MO  or  excusal.  SIR  chits,  same  thing.  There 
is  no  MIDS  module  for  SIR  chits.  1  would  like  to  get  rid  of  the  paper  for  SIR  chits. 

8 

Picture  added  for  each  midshipmen. 

9 

Easier  ability  to  query  by  personally  designated  variables;  Company  statistics  listing  change  in  peformance 
(academics,  athletics,  military  areas,  etc.) 

10 

I'd  like  an  email  sent  to  MIDN  who  are  deliquent  in  PE.  Within  a  week,  they  need  to  respond  to  a  POAto  fix  their 
problem. 

The  MAPR's  portion  is  very  cumbersome  and  can  be  done  via  email. 

I'd  like  one  page  that  says  here  are  all  of  your  issues.. .PE  failures/deficiences/conduct  and/or  honor 
probation/UNSAT/medical  chits/etc. 

Plus  we  need  a  tickler  to  see  all  of  the  awards  or  programs  neededing  our  attention. ..NAPS  detail,  plebe  summer 
detail,  OPINFO,  Kaufman  Award.  Sen  Marg  Smith  Award,  USA  today  Academic  All-American. ...etc... 

11 

Its  great  that  we  pull  everyhting  from  a  real-time  database,  but  1  would  like  to  see  MIDS  work  from  a  cached  copy 
on  my  computer.  The  vast  majority  of  the  data  never  changes.  We  should  use  a  locally  cached  copy  as  the  default 
and  update  it  on  user  request.  This  would  work  well,  since  1  almost  always  look  at  the  records  of  the  same  137  Mids 
(the  ones  in  my  company).  Also,  the  ad  hoc  queries  should  be  simpler  and  more  intuitive  to  construct.  Very  few 
people  know  how  build  a  good  one.  The  laundry  list  of  corporate  documents  is  a  soup  sandwich.  That  mess  should 
be  significantly  streamlined. 

12 

absence  data  more  clearly  shown 

13 

It  would  be  nice  to  be  able  to  keep  a  running  commentary  on  midshipmen  (i.e.  results/problems  discussed  in 
counseling,  performance  notes,  etc). 

14 

More  of  the  often  required  queries  only  accessible  in  AdHoc  System  being  standardized  for  MIDS  6-week  grade 

summaries,  etc.) 

15 

How  Many  Weekends  have  been  used. 

Tours  completed 

16 

A  few  things  would  be  nice  thatl  can  access  from  AdHoc  Queries  (but  not  MIDS),  but  the  inconvenience  is  really 
negligible. 

17 

None 

18 

1  would  like  to  see  the  MO  system  updated  to  be  more  user  friendly.  Mids  often  come  to  see  me  because  they  don't 
know  why  they  were  disapproved  for  a  given  MO. 
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12. 

How  well  does  the  current  MIDS  documentation  (Pocket  Guide, 
online  help,  etc.)  meet  your  needs? 

Miuttboraf 

tResponses 

Responso 

Ratio 

Not  at  all  1. 

8 

A  little  2. 

6 

Somewhat  3. 

6 

Mostly  4. 

4 

16% 

Completely  5. 

1 

4% 

Total 

25 

100% 

13.  How  well  does  current  MIDS  training  support  your  needs? 

Miunboref 

Responses 

Response 

Ratio 

Not  at  all  1. 

rntmamammamm 

14 

58% 

A  little  2. 

mam 

4 

17% 

Somewhat  3. 

mm 

3 

13% 

Mostly  4. 

m 

2 

8% 

Completely  5. 

1 

4% 

Total 

■■ 

100% 
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lii 

What  improvements  can  be  made  in  these  areas  (training  and  documentation)? 

Response 

1 

Provide  ar^  adhocfurtction  that  will  graphically  display  bar  graphs,  pie  graphs,  etc  of  individual  and  company  wide 
information  lOT  indicate/monitor  trends  In  academic,  physical,  athletic  performance. 

2 

Personally,  1  don't  need  training  or  documentation  to  effectively  use  the  MIDS  program.  The  program  is  not 
difficult  to  figure  out  on  your  own.  If  1  run  into  difficulty,  which  is  rare,  1  contact  Mr.  Hawd<i  ns  (Perform  a  nee)  or  Ms. 
Rishell(MIS  Officer). 

3 

There  Is  no  training,  just  a  pocket  guide 

4 

More  training. 

Most  of  my  training  has  been  OJT. 

5 

At  least  some  training. 

6 

#12/#13:  Had  little  or  no  training  on  MIDS.  Need  to  add  "n/a"  to  your  answers,  or"don't  remember." 

7 

I  have  never  had  any  training  in  MIDS.  I  have  seen  one  pamphlet  in  2  years  on  MIDS.  That  was  2  months  ago.  It 
didn't  tell  me  anything  that  I  didn't  already  know. 

8 

I  think  a  basic  training  session  specifying  the  capabilities  of  the  system  would  be  useful.  MIDS  is  very  easy  to 
operate  but  the  AD  HOC  QUERY  function  needs  much  more  training  to  make  it  useful  -  particular  how  do  you  alter 
or  make  new  queries  to  meet  to  your  company's  needs. 

9 

MIDS  training?  what  are  you  talking  about?  we  don't  have  it.  and  frankly,  we  shouldn't. .if  a  LT  in  the  Navy  cant 
get  on  a  computer  and  figure  it  out,  then  we're  sending  the  wrong  LT's  here. ..this  isn't  rocket  science,  ifs  already  a 
user  friendly  computer  system. 

10 

I've  never  used  either 

11 

Have  formal  training  on  its  functionality.  I  learned  by  messing  around  with  certain  modules  until  I  found  out  which 
ones  were  most  useful  for  me.  I  did  not  receive  formal  training  on  Its  functionality  and  all  of  the  different  modules. 

12 

I'm  not  sure  wih at  you  mean...I  don't  use  the  pocket  guide  (never  seen  it)  nor  have  I  ever  really  received  formal  trng 
on  MIDS. 

13 

I  received  almost  no  formal  MIDS  training.  I  taught  myself,  so  really  anything  would  be  an  Improvement.  The 
pocket  guide  is  pretty  good,  and  most  of  the  on-line  help  functions  work  pretty  well. 

14 

adhoc  query  Info  in  MIDS  or  training  on  howto  make  an  adhoc  query  ourselves 

15 

MIDS  is  fairly  intuitive.  What  a  new  Company  Officer/Senior  Enlisted  needs  is  someone  to  walkthrough  the  system 
to  demonstrate  what  data  is  available. 

16 

I  have  don't  recall  any  training  on  the  MIDS  system.  But  it  is  something  that  requires  hands  on  participation.  No 
trng  Is  actually  required. 

17 

I  never  really  received  any  training.  Most  was  OJT.  I  think  it  could  be  more  user  friendly,  but  overall  is  a  good 
system. 
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APPENDIX  B:  PERFORMANCE  MEASUREMENT  PROCEDURES 

This  appendix  outlines  standardized  procedures  for  Company  Officers  to  use  in 
measuring  the  performance  of  their  midshipmen.  The  purpose  of  this  document  is  to 
provide  the  Company  Officer  with  a  collection  of  pre-formatted,  standardized  reports  that 
track  important  performance  metrics.  These  reports  can  be  quickly  and  easily  generated 
as  needed  by  the  individual  Company  Officer.  Some  of  the  procedures  are  applicable  to 
individual  midshipmen,  while  some  are  useful  for  tracking  the  performance  of  the 
company  as  a  whole.  The  procedures  are  grouped  into  five  categories — academics, 
attendance,  conduct,  military  performance,  and  physical  readiness. 

These  procedures  assume  that  the  user  has  Company  Officer-level  access  to 
MIDS  and  Webintelligence,  has  copies  of  the  Webintelligence  query  documents 
referenced  herein,  and  has  a  basic  working  knowledge  of  MIDS,  Webintelligence, 
Microsoft  Excel,  and  Windows. 

Each  procedure  page  consists  of  a  step-by-step  description  of  how  to  generate  the 
report.  This  description  is  accompanied  by  a  sample  screenshot  of  what  the  report  should 
look  like.  In  some  cases,  intermediary  screenshots  are  also  included  where  it  helps  to 
clarify  the  procedural  steps. 
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#  OF  AC  BOARD  CASES  PER  SEMESTER,  COMPARED  TO  REST  OF  THE 
BATTALION: 

1.  Open  Webintelligence  documents  “Ac  Boards  -  Co  vs  Batt”.  When  prompted, 
enter  the  battalion  and  the  beginning  ac  year  of  the  report. 

2.  View/print  documents  as  desired 


'^Document  Results  Ac  Boaids  -  Co  vs.  Ball  -  Miciosoft  Inleinet  Exploits 


Stops  \lfafaah  ^Tlorog  j  Sea^r‘^dvcti»^ 


w. 


'PMntVC^^Oiieu^  ;itetkca^ 


|Ai>^'|€)«atof/EKecUeWIS?^SOefaultRFfamatgn>iy-TQ>s«flepositotyT3pe^ocumenlNjf^^  :  ]|iafer^ 
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#  OF  MIDS  ON  ACADEMIC  PROBATION  PER  SEMESTER,  COMPARED  TO 
REST  OF  THE  BATTALION: 


1.  Open  Webintelligence  documents  “Mids  on  Academic  Probation  vs  Batt”.  When 
prompted,  enter  the  company  information  requested.  (If  desired,  the  trends  for 
just  one  company  can  be  viewed  by  entering  the  same  company  number  at  both 
prompts.) 

2.  View/piint  documents  as  desired. 


I  3  Document  Results  Mids  on  Academic  Probation  vs  Balt  *  Microsolt  Internet  Explorer 


ypesM8jRepositoiyName=8cDocufnentType=W8cD ocumer^ld=£KcUsefSubmjt=&Date=9871 034B2?tFotceRefresli*i0pen=Y  I 


My  Info  View  ^ 


Corporate 
I  Documents 

Personal 
I  Documents 

Inbox 
[  Documents 

Create 
I  Documents 


Search 

Options 

Logout 

Help 


Download  Refresh  Edit  Save  Publish  Send  Delete  < 


iJi  f[iPilTI<ll 


HKSKHniwBII 

HiHBin 


« 

B 

□ 

C 

□ 

D 
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QPR,  WITHIN  COMPANY,  BY  CLASS: 

1.  Open  Webintelligence  documents  “Company  QPR  Trends  (gr^h)”  and/or 
“Company  QPR  Trends  (table)”. 

2.  View/print  documents  as  desired 
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QPR,  COMPANY  COMPARED  TO  THE  REST  OF  THE  BATTALION: 


1.  Open  Webintelligence  document  “Comp  QPR  in  Batt  (table)”.  When  prompted, 
enter  the  company  information  required  and  the  first  academic  year  to  be  included 
in  the  report. 


2.  Select  “Download”  and  save  file  your  local  reports  folder  on  hard  drive,  naming  it 
“Co  QPR  in  Batt”.  File  will  be  saved  as  a  comma-separated  values  (.csv)  file. 

3.  Open  the  downloaded  file  with  Microsoft  Excel.  Save  the  modified  document  as 
an  Excel  file  (File  -  Save  As,  then  select  “Microsoft  Excel  Workbook”  firom  the 
“Save  as  type”  drop  down  list.) 

4.  Select  Pivot  Table  and  Pivot  Chart  Report  firom  the  Data  menu. 

5.  The  PivotTable  Wizard  will  open.  In  the  first  dialog  box,  select  “Microsoft  Excel 
List  or  Database”  and  “PivotChart  (with  PivotTable)”,  then  chck  the  Next  button 

6.  In  the  second  dialog  box,  click  the  Next  button 

7.  In  the  third  dialog  box,  ensure  “New  worksheet”  is  selected,  then  click  on  the 
Layout  button. 

8.  When  the  layout  window  opens,  drag  each  button  on  the  right  to  the  respective 
area  on  the  left  as  listed  below: 

•  “Ac  Year  Ending”  ^  “ROW” 

•  “Semester”  ^  “ROW” 

•  “Avg  Sem  QPR”  ^  “DATA” 

•  “Mid’s  Company  Number”  “COLUMN” 

Now  click  the  Ok  button  to  close  the  layout  window,  followed  by  the 
Finish  button  to  complete  the  PivotTable  Wizard. 

9.  When  tiie  chart  appears,  click  on  the  Chart  Wizard  button  in  the  floating 
PivotTable  toolbar. 
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10.  The  Chart  Wizard  will  open.  In  the  left  pane  (Chart  Type),  select  “Line”.  Then, 
in  the  right  pane  (Chart  Sub-Type)  select  the  top-left  tj^e.  Now,  click  the  Finish 
button  to  close  the  Chart  Wizard. 

11.  Format  the  chart  as  desired,  adding  labels  as  appropriate.  Save  your  work. 
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INDIVIDUAL  QPR  OVER  TIME: 

1.  Open  Webintelligence  document  “Mid  QPR  over  time”  for  the  desired 
midshipmen  (alpha  code  used  for  input).  View/print  graph  as  desired. 
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LIST  OF  MIDS  IN  COMPANY  ON  ACADEMIC  PROBATION: 

1.  From  the  “Company  Officer  -  Summary  Information”  page  in  MIDS,  select  the 
desired  company,  then  check  the  ‘Trobation  Status”  checkbox. 

2.  Press  Find  to  generate  rq)ort. 

3.  View/print  as  desired. 


3  Document  Results  Mids  on  Academic  Probation  vs  Batt  -  Microsoft  Internet  Explore 
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LIST  OF  MIDS  IN  COMPANY  ON  DEAN’S/DANT’S/SUP’S  LIST: 

1.  From  the  “Company  Officer  -  Summary  Information”  page  in  MIDS,  select  the 
desired  company,  then  check  the  “Merit  List  Status”  checkbox. 

2.  Press  Find  to  generate  report. 

3.  View/print  as  desired. 


^  Company  Officer  -  Summary  Informalion  -  Microsoft  Internet  Expic 
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LIST  ALL  ABSENCES  FOR  THE  SEMESTER,  BY  MIDSHIPMAN: 

1.  From  the  “Company  Officer  -  Summary  Mormation”  page  in  MBDS,  select  the 
desired  company,  then  check  the  “Absences:  All”  checkbox. 

2.  Press  Find  to  generate  report. 

3.  View/print  as  desired. 

LIST  OF  TOP  TEN  MIDSHIPMEN  WITH  THE  MOST  ABSENCES: 

1.  From  the  “Company  Officer  ^  Summary  Information”  page  in  MIDS,  select  the 
desired  company,  then  check  the  “Absences:  Top  10”  checkbox. 

2.  Press  Find  to  generate  report. 

3.  View/print  as  desired. 


3  Company  Officer  -  Summary  Information  >  Microsoft  Internet  Explorer 
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COMPANY  VS.  REST  OF  BATTALION  (ALL  ABSENCES/TARDIES): 

1.  Open  Webintelligence  document  “Company  Attendance  vs  Batt”.  When 
prompted,  input  the  company  information  required  and  the  period  for  which  you 
want  to  generate  the  report. 

2.  View/print  graph  as  desired. 
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COMPANY  UA’S  VS.  REST  OF  BATTALION: 

1.  Open  Webintelligence  document  “Company  UAs  vs  Batt”.  When  prompted, 
input  the  company  information  required  and  the  period  for  which  you  want  to 
generate  the  report. 

2.  View/print  graph  as  desired. 


Attendance 


COMPANY,  BY  CLASS  YEAR  (ALL  ABSENCES/TARDIES): 

1.  Open  Webintelligence  document  “Company  Attendance  by  Class”.  When 
prompted,  input  the  company  information  required  and  the  period  for  which  you 
want  to  generate  the  report. 

2.  View/print  graph  as  desired. 


'^Document  Results  Co  Attendance  by  Class  -  Mictosoh  internet  Explorer 
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COMPANY  UA’S: 

1.  Open  Webintelligence  document  “Company  UAs”.  When  prompted,  input  the 
company  information  required  and  the  period  for  which  you  want  to  generate  the 
report. 

2.  View/print  graph  as  desired. 


1  3  Document  Results  Company  UAs  -  Microsoft  Internet  Explorer 
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#  OF  MIDS  IN  COMPANY  COMMITTING  OFFENSES  PER  SEMESTER,  BY 
SEVERITY: 

1.  Open  Webintelligence  document  “Conduct  -  Mids  committing  offenses  per 
semester”.  When  prompted,  input  the  company  information  required. 

2.  View/print  graph  as  desired. 
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individual  conduct  records: 

1.  Bring  up  the  midshipman’s  information  using  the  Company  Officer  System 
module  of  MIDS.  (Search  by  name,  alpha  code,  etc.) 

2.  Individual  PRT  results  are  in  the  sections  of  the  report  titled  “Conduct”  and 
“Conduct  Offenses”. 

3.  Details  for  a  particular  offense  can  be  accessed  by  clicking  on  the  case  number. 
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Records  1  to  8  of8 
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#  OF  MIDS  IN  COMPANY  COMMITTING  OFFENSES  PER  SEMESTER 
COMPARED  WITH  REST  OF  BATTALION: 

1.  Open  Webintelligence  document  “Conduct  -  Major  Offences  vs  Batt”.  When 
prompted,  input  the  company  information  required  and  the  first  ac  year  of  the 
report. 

2.  View/print  graph  as  desired. 


Conduct 


LIST  OF  MIDS  IN  COMPANY  ON  CONDUCT  PROBATION: 

1.  From  the  “Company  Officer  -  Summary  Information”  page  in  MIDS,  select  the 
desired  company,  then  check  the  “Probation  Status”  checkbox. 

2.  Press  Find  to  generate  report. 

3.  View/print  as  desired. 


3  Company  0^^icer  -  Summary  Information  -  Microsoft  Internet  Explorer 


Back  ‘  f<^af4 .  ^  ‘.Stop !> 


favorites,  ‘ 


r <-w«a  ■  '  -•. 

iMail ,  ■  ^  Discuss  Relcom'  .  /f 


>y.usna.edu:801 0/ITSD/mids/DCOWQ004$.STARTUP 


Company  Officer  -  Summary  Information 


Company:;|oi  g 


C  Absences:  Ail  C  Athletic  Status 

C  Absences:  >=  |l  0  C  Conduct  Status 


r  Leave  Status  C  Probation  Status 

C  Merit  List  Status  <5*  pRj  status 

C  Absences:  Top  10  C  eCA  Status  C  Movement  Ordeis  and  Excusals  C  Striper  Status 

C  UA  Tardy:  Top  10  C  General  Information  C  QPRs  and  Standings 


r  Download  THe 


General  Academics  S^onunandant's  Staff 


122 


Military  Performance 


#  OF  D’S/F’S  IN  MILITARY  PERFORMANCE  COMPARED  WITH  THE 
BATTALION: 

1.  Open  Webintelligence  document  ‘Terformance  —  D&F's  in  batt”.  When 
prompted,  input  the  company  information  required  and  the  period  for  which  you 
want  to  generate  the  report. 

2.  View/print  graph  as  desired. 


3  Document  Results  Performance  -  D&F's  in  batt  -  Microsoft  Internet  Explorer 
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LIST  OF  MIDS  IN  COMPANY  ON  PERFORMANCE  PROBATION: 

1.  From  the  “Company  Officer  -  Summary  Information”  page  in  MIDS,  select  the 
desired  company,  then  check  the  “Probation  Status”  checkbox. 

2.  Press  Find  to  generate  report. 

3.  View/print  as  desired. 
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COMPANY  AVERAGE  PRT  SCORES,  COMPARED  TO  THE  REST  OF  THE 
BATTALION: 

1.  Open  Webintelligence  document  “Avg  PRT  scores  in  Batt  (table)”.  When 
prompted,  enter  the  battalion  number  and  the  first  academic  year  to  be  included  in 
the  report. 


2.  Select  “Download”  and  save  file  your  local  reports  folder  on  hard  drive,  naming  it 
“Avg  PRT  scores  in  Batt”.  File  will  be  saved  as  a  comma-separated  values  (.csv) 
file. 

3.  Open  the  downloaded  file  with  Microsoft  Excel.  Save  the  modified  document  as 
an  Excel  file  (File  -  Save  As,  then  select  “Microsoft  Excel  Workbook”  from  the 
“Save  as  type”  drop  down  list.) 

4.  Select  Pivot  Table  and  Pivot  Chart  Report  from  the  Data  menu. 

5.  The  PivotTable  Wizard  will  open.  In  the  first  dialog  box,  select  “Microsoft  Excel 
List  or  Database”  and  “PivotChart  (with  PivotTable)”,  then  click  the  Next  button 

6.  In  the  second  dialog  box,  click  the  Next  button 

7.  In  the  third  dialog  box,  ensure  “New  worksheet”  is  selected,  then  click  on  the 
Layout  button. 

8.  When  the  layout  window  opens,  drag  each  button  on  the  right  to  the  respective 
area  on  the  left  as  hsted  below: 

•  “Ac  Year  Ending”  “ROW” 

•  “Semester”  ^  “ROW” 
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•  “AvgPRT  score”  ^  “DATA” 

•  “Company”  “COLUMN” 

Now  click  the  Ok  button  to  close  the  layout  window,  followed  by  the 
Finish  button  to  complete  the  PivotTable  Wizard. 

9.  When  the  chart  appears,  click  on  the  Chart  Wizard  button  in  the  floating 
PivotTable  toolbar. 

10.  The  Chart  Wizard  will  open.  In  the  left  pane  (Chart  Type),  select  “Line”.  Then, 
in  the  ri^t  pane  (Chart  Sub-Type)  select  the  top-left  type.  Now,  click  the  Finish 
button  to  close  the  Chart  Wizard. 

11.  Format  the  chart  as  desired,  adding  labels  as  appropriate.  Save  your  work. 
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INDIVIDUAL  PRT  RESULTS: 

1.  Bring  up  the  midshipman’s  information  using  the  Company  Officer  System 
module  of  MIDS.  (Search  by  name,  alpha  code,  etc.) 

2.  Individual  PRT  results  are  in  the  section  #7  of  the  report,  “PRT  Grades”. 
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